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ABSTRACT 
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core subject areas-. This report presents the results of the NAEP 2000 
national science assessment of 4th, 8th, and 12th grade students, and state- 
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jurisdictions. Results in 2000 are compared to results from the 1996 science 
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average scores on a 0-300 scale for each grade, and in terms of the 
percentages of students attaining basic, proficient, and advanced achievement 
levels. The achievement levels are performance standards adopted by the 
National Assessment Governing Board (NAGB) as part of its statutory 
responsibilities and describe what students should know and be able to do. 
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What is The Nation’s Report Card? 

THE NATION’S REPORT CARD, the National Assessment ofEducational Progress (NAEP),is the only nationally 
representative and continuing assessment of what America’s students know and can do in various subject areas. Since 
1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and 
other fields.By making objective information on student performance available to policymakers at the national, state, 
and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only 
information related to academic achievement is collected under this program. NAEP guarantees the privacy of 
individual students and their families. 

NAEP IS conducted under the legislative authority of Congress. The Commissioner of the National Center for 
Education Statistics, under the U.S. Department ofEducation, is responsible by law for carrying out the NAEP project 
through competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also 
responsible for providing continuing reviews, including validation studies and solicitation of public comment, on 
NAEP’s conduct and usefulness. 

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines 
for NAEP.The Board is responsible for selecting the subject areas to be assessed from among those included in the 
National Education Goals; for setting appropriate student performance levels; for developing assessment objectives and 
test specifications through a national consensus approach; for designing the assessment methodology; for developing 
guidelines for reporting and disseminating NAEP results; for developing standards and procedures for interstate, 
regional, and national comparisons; for determining the appropriateness of test items and ensuring they are free from 
bias; and for taking actions to improve the form and use of the National Assessment. 
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ixecutive Summary 



The National Assessment of Educational Progress (NAEP) is 
the nation’s only ongoing representative sample survey of 
student achievement in core subject areas. In 2000, NAEP 
conducted a national science assessment of fourth-, eighth-, 
and twelfth-grade students. State-level results were also 
collected at the fourth and eighth grades within 
participating states and jurisdictions. 

Authorized by Congress and administered by the National 
Center for Education Statistics (NCES) in the U.S. 
Department of Education, NAEP regularly reports to the 
pubhc on the educational progress of students in grades 4, 8, 
and 12. This report presents the results of the NAEP 2000 
science assessment for the nation and the states. Results in 
2000 are compared to results from the 1996 science 
assessment. Students’ performance on the assessment is 
described in terms of average scores on a 0-300 scale for 
each grade and in terms of the percentages of students 
attaining three achievement levels: Basic, Proficient, and 
Advanced.The achievement levels are performance standards 
adopted by the National Assessment Governing Board 
(NAGB) as part of its statutory responsibilities and describe 
what students should know and be able to do. The 
Governing Board is an independent bipartisan group created 
by Congress in 1988 to set poHcy for NAEP. 
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As provided by law, the Deputy 
Commissioner of Education Statistics, 
upon review of a congressionally mandated 
evaluation of NAEP, determined that the 
achievement levels are to be considered 
developmental and should be interpreted 
and used with caution. However, both the 
Deputy Commissioner and NAGB believe 
these performance standards are useful for 
understanding trends in student achieve- 
ment. They have been widely used by 
national and state officials as a common 
yardstick of academic performance. 

In addition to providing average scores 
and achievement-level performance at the 
national level and state level, this report 
presents results for subgroups of students 
defined by various background and con- 
textual characteristics. This report also 
contains results for a second sample at both 
the national and state levels — one in which 
testing accommodations were provided to 
students with special needs (i.e., students 
with disabilities or limited English profi- 
cient students). 



The results presented in this report are 
based on representative samples of students 
for the nation and for participating states 
and jurisdictions. In the national sample, 
approximately 47,000 students from 2,100 
schools were assessed. In the state samples, 
approximately 180,000 students from 7,500 
schools were assessed. The national sample 
included students attending both public 
and nonpublic schools, while the state 
samples included only students attending 
public schools. 

A summary of overall results from the 
2000 NAEP science assessment is presented 
on the following pages. Differences be- 
tween results from 1996 and 2000 or 
between groups of students are discussed 
only if they have been determined to be 
statistically significant. 
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Overall Science Results for the 
Nation, Regions, and States 

Science Results for the Mation: 

□ Between 1996 and 2000, there was no 
statistically significant difference observed 
in the average science scores of fourth- or 
eighth-grade students. The average score 
of students in grade 12, however, de- 
clined from 150 in 19.96 to 147 in 2000. 

□ In 2000, the percentage of students 
performing at or above Proficient — 
identified by NAGB as the level that all 
students should reach — was 29 percent 
at grade 4, 32 percent at grade 8, and 18 
percent at grade 12. The percentage of 
eighth-graders at or above Proficient was 
higher in 2000 than in 1996. The per- 
centage of twelfth-graders at or above 
Basic declined between 1996 and 2000. 



and 5 other jurisdictions that participated 
at grade 4, and 38 states and 4 other 
jurisdictions at grade 8. Only public 
schools participated in the state-by-state 
assessment. 

At Grade 4: 

□ The top six states in 2000 were Iowa, 
Maine, Massachusetts, Montana, North 
Dakota, and Vermont. The average scores 
for these six states were higher than any 
other participating state but were not 
found to differ significantly from one 
another. 

□ Iowa, Maine, Massachusetts, Montana, 
and Vermont had percentages of fourth- 
graders at or above Proficient that were 
higher than the other participating states, 
but were not found to be significantly 
different from one another. 



□ The 90th percentile score at grade 8 was 
higher in 2000 than in 1996, indicating 
improvement for the highest-performing 
eighth-graders. At grade 12, the 50th 
percentile score declined between 1996 
and 2000, indicating a decline in the 
performance of middle-performing 
twelfth-graders. 

Science Results for the Regions: 

□ In 2000, the average scores for fourth- 
and eighth-graders were higher in the 
Northeast and Central regions than in 
the Southeast and West. Among twelfth- 
graders, average scores were higher in 
the Northeast and Central regions than 
in the Southeast. 

□ Crade 12 students attending schools in 
the Central region had a lower average 
score in 2000 than in 1996. 

Science Results for the States 
and Other Jurisdictions: 

In the NAEP 2000 state-by-state assess- 
ment, results were reported for 39 states 



O 




At Grade 8: 

□ The top 10 states and other jurisdic- 
tions in 2000 were Idaho, Maine, 
Massachusetts, Minnesota, Montana, 
North Dakota, Ohio, Vermont, and the 
Department of Defense domestic and 
overseas schools.The state of Montana, 
however, had an average eighth-grade 
score that was higher than any other 
participating state or jurisdiction. 

□ Between 1996 and 2000, eighth-graders’ 
average scores increased in Missouri and 
at the Department of Defense domestic 
and overseas schools. (These results are 
based on multiple-comparison statistical 
significance testing procedures including 
all states or jurisdictions that participated 
in both years.) 

□ Massachusetts, Minnesota, Montana, and 
Ohio aU had percentages of eighth- 
graders at or above Proficient that were 
higher than the percentages in other 
participating states, but were not found 
to differ significantly from one another. 
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National Science Results for 
Student Subgroups 

In addition to overall results for the nation 
and for states and jurisdictions, NAEP 
reports on the performance of various 
subgroups of students. Observed differences 
between student subgroups in NAEP 
science performance most likely reflect a 
range of socioeconomic and educational 
factors not addressed in this report or by 
NAEP 

Gender 

O In 2000, males had higher average scores 
than females at grades 4 and 8. The 
apparent gender difference at grade 12 
was not statistically significant. 

n Between 1996 and 2000, the average 
score for eighth-grade males increased, 
while the average score for twelfth-grade 
males decreased. 

n Between 1996 and 2000, the average 
score gap favoring males over females 
widened by three points at grade 4 and 
by five points af grade 8. 

Race/Ethnicity 

n In 2000, the average scores of White 
students at all three grades were higher 
than those of their Black, Hispanic, or 
American Indian peers, and American 
Indian students scored higher on average 
than Black students. 

n Between 1996 and 2000, average scores 
decreased for eighth-grade American 
Indian students and for twelfth-grade 
White students. 

n Between 1996 and 2000, no significant 
difference was observed in the average 
score gap between White and Black 
students and between White and His- 
panic students at any of the three grades. 



Parents’ Level of Education 

□ Generally, students in grades 8 and 12 
who reported higher levels of parental 
education had higher average scores in 
2000 than did their peers who reported 
lower levels of parental education. 
(Information about parental education 
was not collected at the fourth grade.) 

n Between 1996 and 2000, average scores 
dechned among twelfth-graders who 
reported that their parents’ highest level 
of education was high school graduation 
and among those who reported that at 
least one parent had some education 
after high school. 

Type of School 

□ At all three grades in 2000, students 
attending nonpublic schools had higher 
average scores than their peers attending 
pubhc schools. 

G Between 1996 and 2000, the average 
score for twelfth-grade pubHc-school 
students decreased, while the average 
score for twelfth-grade nonpublic-school 
students increased. 

Type of Location 

G In 2000, fourth- and eighth-grade 
students attending schools in central city 
locations had lower average scores than 
their counterparts attending schools in 
urban fringe/large town or rural/small 
town locations. At grade 12, there was 
no statistically significant relationship 
between school location and students’ 
average scores. (Results by type of 
location are not available from 1996.) 
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Free/Reduced-Price School Lunch 
Eligibility 

□ At all three grades in 2000, students 
eligible for the free/ reduced-price 
school lunch program administered by 
the U.S. Department of Agriculture 
(USDA) had lower average scores than 
those who were not eHgible. Free/ 
reduced-price school lunches are in- 
tended for children at, near, or below the 
poverty Hne: eHgibiHty is determined by 
the USDA’s Income EHgibihty Guide- 
lines. (http : / / WWW. fhs . usda . gov /end/ 
lEGs&NAPs/IEGs.htm). 

□ Between 1996 and 2000, the average 
score of eighth-graders who were 
eHgible for free/ reduced-price school 
lunch decreased, while the average score 
of eighth-graders who were not eHgible 
increased. Among twelfth-graders, the 
average score of students who were not 
eligible decreased between 1996 and 
2000. 

Becoming a More 
Inclusive NAEP 

A second set of results from the NAEP 
2000 science assessment includes the 
performance of special-needs students who 
were provided with testing accommoda- 
tions. A similar set of results is available 
from 1996 at the national level only, 
allowing for comparisons between 1996 
and 2000 national results based on 
administration procedures that permitted 
accommodations. 



Science Results for the IMation: 

□ In 2000, the difference between “accom- 
modations-permitted” and “accommo- 
dations-not-permitted” national average 
scores was not found to be statistically 
significant at grades 8 and 12. At grade 4, 
however, the “accommodations-permit- 
ted” average score was 2 points lower 
than the “accommodations-not-permit- 
ted” average score. ^ 

□ Between 1996 and 2000, the national 
average score for twelfth-graders de- 
chned when accommodations were not 
permitted and when accommodations 
were permitted. 

Science Results for the States 
and Other Jurisdictions: 

□ In 2000, none of the apparent differences 
between “accommodations-permitted” 
and“accommodations-not-permitted” 
average scores were found to be statisti- 
cally significant at either grade 4 or 
grade 8 for any of the participating states 
and jurisdictions. (These results are based 
on multiple-comparison statistical 
significance testing procedures including 
all states or jurisdictions that participated 
in 2000.) 
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The effects of offering accommodations are examined in greater detail in two forthcoming reports: 

Lutkus, A. D,, & MazzeoJ. Including special-needs students in the NAEP 1998 reading assessment: Part I, comparison 
overall results with and without accommodations. W 2 ishinp:on, DC: US. Department of Education, Institute of 
Education Sciences, National Center for Education Statistics (forthcoming). 

Lutkus,A. D Including special-needs students in the NAEP 1998 reading assessment: Part II, results for students with 
disabilities and limited English proficient students. Washington, DC: U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Statistics (forthcoming). 
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School Contexts 
for Learning Science 

NAEP collects information about the 
contexts for student learning by adminis- 
tering questionnaires to assessed students, 
their teachers, and school administrators. 
Using the student as the unit of analysis, 
NAEP examines the relationship between 
selected contextual variables drawn from 
these questionnaires and students’ average 
scores on the science assessment. In inter- 
preting these data, readers are reminded 
that the relationship between contextual 
variables and student performance is not 
necessarily causal. There are many factors 
that may play a role in student performance 
on NAEP. 



Grade 4; 

Computer Availability and Use 

n In 2000, fourth-graders whose teachers 
reported that they used computers for 
science instruction scored higher, on 
average, than fourth-graders whose 
teachers reported that they did not. 

n Between 1996 and 2000, the percentage 
of fourth-graders whose teachers re- 
ported using computers for science 
instruction increased from 47 to 57 
percent. 

Coursework 

□ In 2000, fourth-graders whose teachers 
reported spending a lot of time or some 
time on life science and Earth science 
had higher average scores than fourth- 
graders whose teachers reported 
spending only a little time on these 
domains. 

□ In 2000, 31 percent of fourth-grade 
students were taught by teachers who 
reported spending a lot of time on life 
science and Earth science, and 22 per- 
cent were taught by teachers who 
reported spending a lot of time on 
physical science. 

n Between 1996 and 2000, the percentage 
of fourth-graders whose teachers re- 
ported spending a lot of time on Earth 
science increased from 19 to 31 percent. 
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Grade 8: 

Computer Availability and Use 

□ In 2000, eighth-graders whose science 
teachers reported having their students 
use computers for simulations and 
modeling or for data analysis and other 
applications had higher average scores 
than eighth-graders whose science 
teachers reported not having students 
use computers in this manner. 

□ Between 1996 and 2000, the percentage 
of eighth-graders whose science teachers 
reported having their students use 
computers for data analysis and other 
applications or for word processing 
increased. 

Coursework 

□ In 2000, 45 and 47 percent of eighth- 
graders were taught by teachers who 
reported spending a lot of time on Earth 
science and physical science, respectively. 
Twenty-one percent of eighth-graders 
were taught by teachers who reported 
spending a lot of time on life science. 

Grade 12: 

Computer Use 

□ In 2000, twelfth-graders who reported 
using computers to collect data or to 
analyze data in their science classes once 
a month or more had higher average 
scores than twelfth-graders who re- 
ported doing so less frequently. 



□ In 2000, twelfth-graders who reported 
never downloading data and related 
information from the Internet for their 
science classes had lower average scores 
than twelfth-graders who reported doing 
so at least sometimes. 

Coursework 

□ Twelfth-graders who reported that they 
were currently taking a science course in 
2000 scored higher, on average, than 
twelfth-graders who reported that they 
were not. 

□ According to twelfth-graders’ reports in 
2000 about the types of science courses 
they had taken since eighth-grade, 
approximately 74 percent had taken 
Earth science, 92 percent had taken 
biology, 70 percent had taken chemistry, 
and 36 percent had taken physics. 

□ Twelfth-grade students who reported in 
2000 that they had taken or were cur- 
rently enrolled in Advanced Placement 
(AP) biology, chemistry, or physics had 
higher average scores than twelfth-grade 
students who said they had not taken 
and were not enrolled in these AP 
courses. 



The full set of results is available in an interactive database on the NAEP web site, 

http://nces.ed.gov/nationsreportcard 

Released test questions from the 1996 and 2000 science assessments 
and question-level performance data are also available on the web site. 
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NAEP 2000 Science Assessment 




What is the 
NAEP science 
assessment? 

How does the 
NAEP science 
assessment 
measure and 
report student 
progress? 



Introduction 

National and international concern for students’ 
achievement in science has been the impetus for several 
recent large-scale efforts to measure science knowledge and 
skills. For example, a repeat of the Third International 
Mathematics and Science Study (TIMSS-R.) was conducted 
in 38 countries in 1999, 26 of which had also participated in 
1995.^ This assessment, conducted under the auspices of the 
International Association for the Evaluation of 
Educational Achievement (lEA),was given to eighth- 
graders and measured students’ knowledge and skills 
in the areas of mathematics and science. In 2000, the 
Program for International Student Assessment (PISA), 
organized by the Organization for Economic 
Cooperation and Development (OECD), was given 
to 15-year-olds in 32 countries for the first time.^ 
This series of tests assessed reading literacy, 
mathematics literacy, and scientific literacy and was 
designed to measure the functional skills that students 
have acquired as they near the end of mandatory 
schooling. In addition to these assessments, in 2000 
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^ Martin, M. O., MuUis, LV. S., Gonzalez, E.J., Gregory, K. D., Smith, TA., Chrostowski, 

S. J., Garden, R.A.,& O’Connor, K. M. (2000). TIMSS 1999 international science report: 
Findings from ISA's repeat of the Third International Mathematics and Science Study at the 
eighth grade. Chestnut Hill, MA: International Study Center, Lynch School ofEducation, 
Boston College. 

Gonzales, P, Calsyn, C., Jocelyn, L., Mak, K., Kastberg, D., Arafeh, S., Williams, T, & 
Tsen,W. (2000). Pursuing excellence: Comparisons of international eighth^grade mathematics 
and science achievement from a U.S. perspective , 1995 and 1999 (NCES Publication No. 
2001-028). Washington, DC: US. Department ofEducation, Office of Educational 
Research and Improvement, National Center for Education Statistics. 

^ Lemke, M., Calsyn, C., Lippman, L., Jocelyn, L., Kastbei^, D, Liu,Y, Roey, S., Williams, 

T, Kruger, T, & Bairu, G. (2000). Outcomes of learning: Results from the 2000 program for 
international student assessment of 15 -year-olds in reading, mathematics, and science literacy (NCES 
Publication No. 2002-1 15). Washington, DC: US. Department ofEducation, Office of 

PI I Educational Research and Improvement, National Center for Education Statistics. 
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the National Assessment of Educational 
Progress (NAEP) administered assessments 
in science and mathematics to students at 
grades 4, 8, and 12, and in reading to 
students at grade 4. A number of states and 
other jurisdictions also took part in the 
science and mathematics assessments at 
grades 4 and 8. As with theTIMSS-R 
assessment, the NAEP assessments were 
designed to measure knowledge and skills 
in the various content domains. 

The results of theTIMSS-R study and 
PISA study provide valuable information 
about the achievement of students in the 
United States vis-a-vis their counterparts 
throughout the world. For example, 
TIMSS-R showed that the science perfor- 
mance of eighth-graders in the U.S. was at 
the international average and no significant 
change was detected since the first TIMSS 
administration in 1995, and the PISA study 
showed that 15-year-olds in the U.S. 
performed at an average level in science 
literacy when compared to students in 
other countries.^ 

A voluntary Benchmarking Study was 
included as part of TIMSS 1999 that 
allowed the participating U.S. states and 
districts or consortia to assess the achieve- 
ment of their students in an international 
context. Of the 13 states that participated 
in the study, all but 3 performed above the 



international average in science.^ 

The results of the NAEP 2000 science 
assessment provide important information 
about the performance of students in the 
nation, states, and other jurisdictions. This 
report discusses these results. It summarizes 
student achievement, compares results from 
the nation, states, and other jurisdictions, 
and discusses some of the many contextual 
variables collected during administration. 

In addition, the report also compares, 
where appropriate, results from the 1996 
and 2000 science assessments. These results 
add to the body of information obtained 
from studies such as TIMSS-R and PISA 
about what students know and can do. 

The results also provide educators and 
policymakers with information that can be 
used to ascertain the well-being of science 
education in the U.S. 

Overview of the 2000 National 
Assessment of Educational 
Progress (NAEP) 

In 1969, NAEP was authorized by Con- 
gress to collect, analyze, and report reliable 
and valuable information about what 
American students know and can do in 
core subject areas. Since that time, in what 
has come to be referred to as the “long- 
term trend assessment,” NAEP has assessed 
public- and nonpublic-school students who 
are 9, 13, and 17 years old. Since 1990, the 



^ Martin, M. O., Mullis, I.V S., Gonzalez, E. J., Gregory, K. D., Smith, T. A., Chrostowski, S. J., Garden, R. A., & 

O Connor, K, M. (2000). TIMSS 1999 international science report: Findings from lEA's repeat of the Third International 
Mathematics and Science Study at the eighth grade. Chestnut Hill, MA: International Study Center, Lynch School of 
Education, Boston College. 

Lemke, M., Calsyn, C., Lippman, L., Jocelyn, L., Kastberg, D., Liu,Y, Roey, S., Williams, T., Kruger,T, & Bairu, G. 
(2000). Outcomes of learning: Results from the 2000 program for international student assessment of 15-year-olds in reading, 
mathematics, and science literacy (NCES Publication No. 2002-1 15). Washington, DC: U.S. Department of Education, 
Office of Educational Research and Improvement, National Center for Education Statistics. 

Gonzales,?., Calsyn, C., Jocelyn, L., Mak, K., Kastberg, D, Arafeh, S., Williams, T., &Tsen,W. (2000). Pursuing 
excellence: Comparisons of international eighth-grade mathematics and science achievement from a U.S. perspective, 1995 and 
1999 (NCES Publication No. 2001 >028). Washington, DC: U.S. Department of Education, Office of Educational 
Research and Improvement, National Center for Education Statistics. 

^ Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., O’Connor, K.M., Chrostowski, S.J., Gregory, K.D, Smith,T.A., & 
Garden, R.A. (2001). Science benchmarking report: TIMSS 1999 — eighth grade. Chestnut Hill, MA; International Study 
^ Center, Lynch School of Education, Boston College. 
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more recently developed assessments, 
referred to as the main NAEP, have also 
assessed public- and nonpublic-school 
students in grades 4, 8, and 12. The results 
provided in this report from the 2000 
science assessment are not comparable to 
those obtained from the 1999 science 
long-term trend assessment due to differ- 
ences in the content of the two assessments, 
as well as different sampling and adminis- 
tration procedures. 

AH NAEP assessments are based on 
frameworks developed through a national 
consensus process. The 2000 NAEP science 
assessment was the second administration 
of an assessment based on The NAEP 
Science Framework^ In 1996 and 2000, the 
NAEP science assessment was administered 
to national samples of fourth-, eighth-, and 
twelfth-graders. The science assessment was 
also administered to samples of eighth- 
graders participating in the state-by-state 
assessment in 1996 and 2000 and to 
samples of fourth-graders participating in 
the state-by-state assessment in 2000. The 
legislation authorizing NAEP did not 
include state-by-state testing in grade 12.^ 

This report describes the results of the 
2000 NAEP science assessment at grades 4, 
8, and 12 and, where appropriate, compares 
results in 2000 to those in 1996. Compari- 
sons across assessment years are possible 
because the assessments were developed 
under the same framework and share a 
common set of science questions, and 
because the populations of students in both 
assessments were sampled and assessed 
using comparable procedures. 



The Science Assessment 
Framework 

The NAEP Science Framework ’’ provided 
the operational specifications and theoreti- 
cal basis for developing NAEP science 
assessments in 1996 and 2000. It was 
developed in 1991 through a consensus 
process involving educators, policymakers, 
science teachers, representatives of the 
business community, assessment and cur- 
riculum experts, and members of the 
public. The project was managed by the 
Council of Chief State School Officers 
(CCSSO) under the auspices of the Na- 
tional Assessment Governing Board 
(NAGB). 

The framework is organized along a 
content dimension and a cognitive dimen- 
sion (knowing and doing). The content 
dimension is divided into three major fields 
of science: Earth, physical, and life. Science 
content pertaining to physics and chemis- 
try is assessed within the field of physical 
science. The cognitive domain is divided 
into conceptual understanding, scientific 
investigation, and practical reasoning. Each 
question in the assessment is categorized by 
its content and cognitive domains. The 
framework also specifies two overarching 
categories — the nature of science and the 
organizing themes of science. Figure 1.1 
summarizes the structure of the 1996 and 
2000 assessments. The framework also 
specifies the percentage of assessment time 
to be devoted to each content and cogni- 
tive domain. A fuller description of the 
framework and a breakdown of the distri- 
bution of assessment time can be found in 
appendix A. 




^ National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of 
Educational Pro^re^i. Washington DC: Author. (Also available online at http://www.nagb.org/pubs/) 

^ Public Law 100-297. (1988). National Assessment of Education Improvement Act (20 USC 1211). 

^ National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of 
Educational Progress, Washington DC: Author. (Also available online at http://www.nagb.org/pubs/) 
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Figure 1.1; Structure of the 2000 Assessment 
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Conceptual 
Understanding 



Scientific 

Investigation 



Practical 

Reasoning 



Fields of Science 

Earth Physical Life 




SOURCE: National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of Educational Progress. 
Washington, DC: Author. 



The Science Assessment 
Instruments 

As the only federally mandated ongoing 
assessment of student science achievement 
on a national scale, it is imperative that 
NAEP reflect the framework and expert 
perspectives and opinions about science 
and its measurement. To that end, the 
assessment development process involves 
reviews by teachers and teacher educators 
as well as by state officials and measure- 
ment experts. All components of the 
assessment are evaluated for curricular 
relevance, developmental appropriateness, 
and fairness. 



The 2000 science assessment booklets at 
grades 4, 8, and 12 consisted of two sepa- 
rately timed sections (i.e. blocks) of science 
questions that included both multiple- 
choice questions and constructed-response 
questions requiring students to create a 
written response. At the fourth grade, 20 
minutes were allowed for each section of 
questions and at the eighth and twelfth 
grades, 30 minutes. In addition, one-half of 
the students in each school sample con- 
ducted a hands-on task and answered 
questions related to the task. For this, too, 
the time allotted was 20 minutes at grade 4 
and 30 minutes at grades 8 and 12. It 
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should be noted that students only took a 
portion of the assessment — two or three 
sections of the 14 sections that comprise 
the whole assessment at grades 4, 8, and 12. 
In addition to the science questions that 
students answered, they also responded to 
background questions that asked them to 
give information about themselves and 
their school experiences. For example, 
students were asked how much time they 
spent on homework, how often they used a 
computer, and what science subjects they 
were currently taking in school. 

Additional information about the design 
of the 2000 science assessment is presented 
in appendix A. 

Description of School 
and Student Samples 

The NAEP 2000 science assessment was 
conducted nationally at grades 4, 8, and 12 
and state -by-state at grades 4 and 8. The 
national assessment included representative 
samples of both public and nonpubhc 
schools. The state -by-state assessments 
included only public schools. In total, 
47,000 students from 2,100 schools were 
assessed in the national sample and 180,000 
students from 7,500 schools in the state 
samples. Additional information about 
school and student samples is given in 
appendix A. 



Jurisdictions including 40 states and 5 
other jurisdictions participated in the state- 
by-state 2000 science assessment at grade 4 
and 39 states and 5 other jurisdictions 
participated at grade 8. The 5 other juris- 
dictions that participated were American 
Samoa, the Department of Defense Do- 
mestic Dependent Elementary and Sec- 
ondary Schools (DDESS), the overseas 
Department of Defense Dependents 
Schools (DoDDS), Guam, and the Virgin 
Islands. To ensure comparability across 
jurisdictions, NCES has established guide- 
hnes for school and student participation 
rates. Appendix A highlights these guide- 
lines, and jurisdictions failing to meet them 
are noted in the tables and figures that 
present the state-by-state results. 

Figure 1.2 fists the jurisdictions that 
participated in the 2000 science assessment 
and notes those jurisdictions that failed to 
meet one or more NCES-established 
participation rate guidelines for public 
schools. Results are not reported for the 
jurisdictions that failed to meet the initial 
school participation rate of 70 percent. 
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Figure 1.2 


Participating jurisdictions in the NAEP 2000 state assessment program in science 


Grade 4 


Alabama 


Louisiana 


New York^ 


Virginia 




Arizona 


Maine^ 


North Carolina 


West Virginia 




Arkansas 


Maryland 


North Dakota 


Wisconsin' 




California^ 


Massachusetts 


Ohio2 


Wyoming 




Connecticut 


Michigan^ 


Oklahoma 


American Samoa 




Georgia 


Minnesota^ 


Oregon^ 


DDESS 




Hawaii 


Mississippi 


Rhode Island 


DoDDS 




Idaho^ 


Missouri 


South Carolina 


Guam 




Illinois^ 


Montana^ 


Tennessee 


Virgin Islands 




Indiana^ 


Nebraska 


Texas 






lowa^ 


Nevada 


Utah 






Kentucky 


New Mexico 


Vermont^ 




Grade 8 


Alabama 


Maine^ 


North Carolina 


West Virginia 




Arizona^ 


Maryland 


North Dakota 


Wisconsin' 




Arkansas 


Massachusetts 


Ohio 


Wyoming 




California^ 


Michigan^ 


Oklahoma 


American Samoa 




Connecticut 


Minnesota^ 


Oregon^ 


DDESS 




Georgia 


Mississippi 


Rhode Island 


DoDDS 




Hawaii 


Missouri 


South Carolina 


Guam 




Idaho^ 


Montana^ 


Tennessee 


Virgin Islands' 




Illinois^ 


Nebraska 


Texas 






Indiana^ 


Nevada 


Utah 






Kentucky 


New Mexico 


Vermont^ 




L ^ 


Louisiana 


New York^ 


Virginia 





^ Failed to meet the initial school participation rate of 70 percent; results not reported. 

^ Failed to meet one or more participation rate guidelines; results reported with appropriate notation. 

For more details on participation rate guidelines, see appendix A. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools 
DoDDS: Department of Defense Dependents School (Overseas) 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 



Two Sets of NAEP Results: 
Accommodations Not Permitted and 
Accommodations Permitted 

The NAEP assessments have always 
sought to include special>>needs students — 
students with disabilities (SD) and limited 
English proficient students (LEP) — to the 
fullest degree possible. However, there have 
always been some special-needs students 
who were excluded from taking the NAEP 
assessment because they could not partici- 



pate meaningfully in the assessment. 
Schools that participate in NAEP have 
been permitted to exclude some students 
who may have Individualized Education 
Programs (lEPs) or are receiving services 
under section 504 of the Rehabilitation act 
of 1973.^ Similarly, schools have been 
permitted to exclude students they identify 
as being LEP Schools are encouraged to 
make exclusion decisions in accordance 
with explicit criteria provided by NAEP 
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In order to move its assessments toward 
more inclusive samples, NAEP began to 
explore the use of accommodations, or 
alternate testing situations, with special- 
needs students in the 1996 science and 
mathematics assessments. This shift toward 
greater inclusiveness allowed NAEP to 
more closely approximate state and district 
testing pohcies that have increasingly 
offered testing accommodations to special- 
needs students. In 1996, the national NAEP 
sample was spUt so that some of the schools 
sampled were permitted to provide accom- 
modations to special-needs students and 
the others were not.This split-sample 
design made it possible to study the effects 
on NAEP results of including special-needs 
students in the assessments under alternate 
testing conditions. A series of technical 
research papers has been published with 
the results of these comparisons.^ Based on 
the outcomes of these technical analyses, 
the 1998 results of those NAEP assessments 
that used new test frameworks (writing and 
civics), and hence also began new trend 
lines, were reported for the first time with 
the inclusion of data from accommodated 
special-needs students. 



This report includes two different sets of 
NAEP results based on the split-sample 
design: 

□ results based on a less inclusive sample 
that did not offer accommodations to 
special-needs students, and 

□ results based on a more inclusive sample 
that did offer accommodations (such as 
extended time and smaU-group adminis- 
tration) to special-needs students. 
Although accommodated students make 

up a small proportion of the total weighted 
number of students assessed (see table A.9 
in appendix A, for details), making accom- 
modations available to special-needs stu- 
dents may change the overall assessment 
results in subtle ways. For example, some 
special-needs students who might have 
been tested without accommodations in 
previous assessment years might now 
receive accommodations, and, possibly, 
attain higher scores. Further, special-needs 
students who might have been excluded in 
previous years might now be included, but 
attain relatively low scores. The findings on 
results when accommodated special-needs 
students were included in the NAEP 
assessment are presented in chapter 4 of 
this report. In addition, appendix A con- 
tains a more detailed discussion of NAEP s 
intent to assess all students from the target 
population. 



^ Olsen, J. F, & Goldstein, A. A. (1997). The inclusion of students with disabilities and limited English proficient students in 
large-scale assessments: A summary of recent progress (NCES Publication No. 97— 482). Washington, DC: U.S. Depart- 
ment of Education, Office of Educational Research and Improvement, National Center for Education Statistics. 

Mazzeo,J., Carlson,]. E.,Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of special-needs students in 
NAEP: A report on 1996 research activities (NCES Publication No. 2000-473). Washington, DC: U.S. Department of 
Education, Office of Educational Research and Improvement, National Center for Education Statistics. 

Lutkus, A. D., & Mazzeo, J. Including special-needs students in the NAEP 1998 reading assessment: Part /, comparison of 
overall results with and without accommodations. Washington, DC: U.S. Department of Education, Institute of Educa- 
tion Sciences, National Center for Education Statistics (forthcoming). 

Lutkus, A. D. Including special-needs students in the NAEP 1998 reading assessment: Part II, results for students with 
disabilities and limited English proficient students. Washington, DC: U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Statistics (forthcoming). 
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Reporting the Assessment Results 

The results of student performance on the 
NAEP science assessment are presented in 
two ways: one, as average scores on the 
NAEP composite science scale and two, as 
the percentage of students attaining NAEP 
science achievement levels. The average 
scale score represents students’ performance 
on the assessment. The achievement levels 
reflect goals for student performance, and 
the percentage of students at each achieve- 
ment level indicate the extent to which 
students are meeting those goals. 

The composite scale at each grade 
ranges from 0 to 300. While the scale score 
ranges are identical, the scale was derived 
independently at each grade. Also scales 
were weighted differently at different 
grades in determining the overall scale. 
Therefore, average scale scores across grades 
cannot be compared. For example, equal 
scale scores on the grade 4 and grade 8 
scales do not imply equal levels of science 
achievement. A full description of NAEP 
scales and scaling procedures can be found 
in the NAEP 2000 Technical Report2^ 

Achievement-level results are presented 
in terms of science achievement levels as 
authorized by the NAEP legislation and 
adopted by the National Assessment Gov- 
erning Board (NAGB)." For each grade 
assessed, NAGB has adopted three achieve- 
ment levels: Basic, Proficient, and Advanced. 



For reporting purposes, the achievement 
cut scores are placed on the science scale, 
resulting in four ranges: below Basic, Basic, 
Proficient, and Advanced. 

The Setting of 
Achievement Levels 

The 1988 NAEP legislation that created 
the National Assessment Governing Board 
directed the Board to identify "appropriate 

achievement goals for each subject 

area” that NAEP measures.'^ The 1994 
NAEP reauthorization reaffirmed many of 
the Board’s statutory responsibilities in- 
cluding “developing appropriate student 
performance standards for each age and 
grade in each subject area to be tested 
under the National Assessment.”'^ In order 
to follow this directive and achieve the 
mandate of the 1988 statute to “improve 
the form and use of NAEP results,” the 
Board undertook the development of 
student performance standards called 
“achievement levels.” Since 1990 the 
Board has adopted achievement levels in 
mathematics, reading, U.S. history, world 
geography, science, writing, and civics. 

The Board defined three levels for each 
grade: Basic, Proficient, and Advanced.The 
Basic level denotes partial mastery of 
prerequisite knowledge and skills that are 
fundamental for proficient work at a given 
grade. The Proficient level represents solid 



National Center for Education Statistics. NAEP 2000 technical report. Washington, DC: Author (forthcoming) 
" Public Law 100-297. (1988). National Assessment of Educational Progress Improvement Act (20USC 1211). 
Public Law 102-382. (1994). Improving America’s Schools Act (20USC 9010). 

Public Law 100-297. (1988). National Assessment of Educational Progress Improvement Act (20USC 1211). 
Public Law 102-382. (1994). Improving America’s Schools Act (20USC 9010). 
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academic performance for each grade 
assessed. Students reaching this level dem- 
onstrate competency over challenging 
subject matter. The Advanced level signifies 
superior performance at a given grade. 
Furthermore, for each grade, the levels are 
cumulative; that is, abilities achieved at the 
Proficient level presume mastery of abihties 
associated with the Basic level, and attain- 
ment of the Advanced level presumes mas- 
tery of both the Basic and Proficient levels. 



Figure 1.3 presents the poficy definitions of 
the achievement levels that apply across all 
grades and subject areas. Adopting three 
levels of achievement for each grade signals 
the importance of looking at more than 
one standard of performance. The Board 
believes, however, that aU students should 
reach the Proficient level; the Basic level is 
not the desired goal, but rather represents 
partial mastery that is a step towards 
Proficient. 



Figure 1 .3 Policy definitions of the three achievement levels 

Achievement Levels 



Basic This level denotes partial mastery of prerequisite knowledge and skills that are 
I fundamental for proficient work at each grade. 

Pnficient 

• i 

i 

i i 

I I 

Advanced j 



This level represents solid academic performance for each grade assessed. Students 
reaching this level have demonstrated competency over challenging subject matter, 
including subject-matter knowledge, application of such knowledge to real-world 
situations, and analytical skills appropriate to the subject matter. 

This level signifies superior performance. 



SOURCE: National Assessment Governing Board. 
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The achievement levels in this report 
were arrived at somewhat differently from 
those adopted by the Board for other 
subject areas. A standard-setting process was 
carried out by ACT, Inc., under contract to 
the Board. ACT convened a cross section 
of educators and interested citizens across 
the nation and asked them to judge what . 
students should know and be able to do 
relative to the body of content reflected in 
the NAEP framework for science. The 
achievement levels arrived at by this pro- 
cess were examined by the Board. In 
several cases, the levels appeared to be set 
either lower or higher than would be 
reasonable, resulting in too few or too 
many students placing at or above the Basic, 
Proficient, or Advanced levels. This belief was 
based on information about eighth-grade 
students from achievement levels adopted 
for other NAEP subjects, 1996 Advanced 
Placement (AP) results for twelfth-graders, 
and information about eighth-grade 
students from the Third International 
Mathematics and Science Study (TIMSS). 
The Board, therefore, adjusted the cut 
scores of some of the levels. Since the 
content descriptions developed by the 



act panehsts no longer matched the cut 
scores adopted by the Board, a second 
panel of science educators and scientists 
was convened to develop new descriptions. 
These descriptions were based on student 
performance at each achievement level, and 
are a measure of what students know and 
can do. The new cut scores and content 
descriptions were adopted by the Board in 
1996.’^ 

Achievement-Level Descriptions 
for Each Grade 

The achievement-level descriptions for 
grades 4, 8, and 12 are presented in flgures 
1.4 through 1.6. As noted previously, the 
achievement levels are cumulative. There- 
fore, students performing at the Proficient 
level also display the competencies associ- 
ated with the Basic level, and students at the 
Advanced level also demonstrate the knowl- 
edge and skills associated with both the 
Basic and Proficient levels. For each achieve- 
ment level listed in figures 1.4 through 1.6, 
the scale score that corresponds to the 
beginning of that level is shown in paren- 
theses. For example, in flgure 1 .4 the scale 
score of 138 corresponds to the beginning 
of the grade 4 Basic level of achievement. 
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Figure 1.4 
Grade 4 



NAEP Science Achievement Levels 



Sssic 

U38) 



Proficient 

(170) 



Advanced 

(205) 



Students performing at the Basic level demonstrate some of the knowledge and reasoning 
required for understanding Earth, physical, and life sciences at a level appropriate to 
grade 4. For example, they can carry out simple investigations and read uncomplicated 
graphs and diagrams. Students at this level also show a beginning understanding of 
classification, simple relationships, and energy. 

Fourth-grade students performing at the Sas/c level are able to follow simple procedures, 
manipulate simple materials, make observations, and record data. They are able to read 
simple graphs and diagrams and draw reasonable but limited conclusions based on data 
provided to them. These students can recognize appropriate experimental designs, although 
they are unable to justify their decisions. 

When presented with diagrams, students at this level can identify seasons; distinguish 
between day and night; and place the position of the Earth, sun, and planets. They are able to 
recognize major energy sources and simple energy changes. In addition, they show an 
understanding of the relationships between sound and vibrations. These students are able to 
identify organisms by physical characteristics and group organisms with similar physical 
features. They can also describe simple relationships among structure, function, habitat, life 
cycles, and different organisms. 

Students performing at the Proficient \ew\ demonstrate the knowledge and reasoning 
required for understanding of the Earth, physicai and iife sciences at a ievei appropriate 
to grade 4. For exampie, they understand concepts relating to the Earth's features, 
physicai properties, structure, and function. In addition, students can formuiate solutions 
to famiiiar problems as well as show a beginning awareness of issues associated with 
technoiogy. 

Fourth-grade students performing at the Proficient level are able to provide an explanation of 
day and night when given a diagram. They can recognize major features of the Earth’s surface 
and the impact of natural forces. They are also able to recognize water in its various forms in 
the water cycle and can suggest ways to conserve it. These students recognize that various 
materials possess different properties that make them useful. Students at this level are able 
to explain how structure and function help living things survive. They have a beginning 
awareness of the benefits and challenges associated with technology and recognize some 
human effects on the environment. They can also make straightforward predictions and 
justify their position. 

Students performing at the Advanced\eve\ demonstrate a solid understanding of the 
Earth, physicai, and life sciences as weil as the abiiity to apply their understanding to 
practicai situations at a Ievei appropriate to grade 4. For example, they can perform and 
critique simple investigations, make connections from one or more of the sciences to 
predict or conclude, and apply fundamental concepts to practical applications. 

Fourth-grade students performing at the /Irfi/a/jcerf level are able to combine information, 
data, and knowledge from one or more of the sciences to reach a conclusion or to make a vaiid 
prediction. They can also recognize, design and explain simple experimental procedures. 

Students at this level recognize nonrenewable sources of energy. They also recognize that light 
and sound travel at different speeds. These students understand some principles of ecology and 
are able to compare and contrast life cycles of various common organisms. In addition, they 
have a developmental awareness of the benefits and challenges associated with technology. 



SOURCE: National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of Educational Progress. 
Washington, DC: Author. 
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Figure 1.5 
Grade 8 



NAEP Science Achievement Levels 



I 




Sasic 

(143) 



Pnfkmt 

(170) 



Students performing at the das/c level demonstrate some of the knowledge and reasoning 
required for understanding of the Earth, physical, and life sciences at a level appropriate 
to grade 8. For example, they can carry out investigations and obtain information from 
graphs, diagrams, and tables. In addition, they demonstrate some understanding of 
concepts relating to the solar system and relative motion. Students at this level also have 
a beginning understanding of cause-and-effect relationships. 

Eighth-grade students performing at the Basic level are able to observe, measure, collect, 
record, and compute data from investigations. They can read simple graphs and tables and 
are able to make simple data comparisons. These students are able to follow directions and 
use basic science equipment to perform simple experiments. In addition, they have an 
emerging ability to design experiments. 

Students at this level have some awareness of causal relationships. They recognize the 
position of planets and their movement around the sun and know basic weather-related 
phenomena. These students can explain changes in position and motion such as the 
movement of a truck in relation to that of a car. They also have an emerging understanding of 
the interrelationships among plants, animals, and the environment. 

Students performing at the Proficient level demonstrate much of the knowledge and many 
of the reasoning abilities essential for understanding of the Earth, physical, and life 
sciences at a level appropriate to grade 8. For example, students can interpret graphic 
information, design simple investigations, and explain such scientific concepts as energy 
transfer. Students at this level also show an awareness of environmental issues, especially 
those addressing energy and pollution. 

Eighth-grade students performing at the Proficient \e\ie\ are able to create, interpret, and 
make predictions from charts, diagrams, and graphs based on information provided to them 
or from their own investigations. They have the ability to design an experiment and have an 
emerging understanding of variables and controls. These students are able to read and 
interpret geographic and topographic maps. In addition, they have an emerging ability to use 
and understand models, can partially formulate explanations of their understanding of 
scientific phenomena, and can design plans to solve problems. 

Students at this level can begin to identify forms of energy and describe the role of energy 
transformation in living and nonliving systems. They have knowledge of organization, gravity, 
and motion within the solar system and can identify some factors that shape the surface of 
the Earth. These students have some understanding of properties of materials and have an 
emerging understanding of the particulate nature of matter, especially the effect of tempera- 
ture on states of matter. They also know that light and sound travel at different speeds and 
can apply their knowledge of force, speed, and motion. These students demonstrate a 
developmental understanding of the flow of energy from the sun through living systems, 
especially plants. They know that organisms reproduce and that characteristics are inherited 
from previous generations. These students also understand that organisms are made up of 
cells and that cells have subcomponents with different functions. In addition, they are able to 
develop their own classification system based on physical characteristics. These students can 
list some effects of air and water pollution as well as demonstrate knowledge of the advan- 
tages and disadvantages of different energy sources in terms of how they affect the environ- 
ment and the economy. 
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Figure 1 .5 NAEP Science Achievement Levels 

Grade 8 
(continued) 



I 



Advanced \ Students performing at the Advanced\BVB\ demonstrate a solid understanding of the 
(208) i Earth, physical, and life sciences as well as the abilities required to apply their 

understanding in practical situations at a level appropriate to grade 8. For example, 

I students can perform and critique the design of investigations, relate scientific concepts 
I to each other, explain their reasoning, and discuss the impact of human activities on the 
I environment. 



Eighth-grade students performing at the Advanced level are able to provide an explanation for 
scientific results. They have a modest understanding of scale and are able to design a 
controlled experiment. These students have an understanding of models as representations of 
natural systems and can describe energy transfer in living and nonliving systems. 



I 




Students at this level are able to understand that present physical clues, including fossils 
and geological formations, are indications that the Earth has not always been the same and 
that the present is a key to understanding the past. They have a solid knowledge of forces and 
motions within the solar system and an emerging understanding of atmospheric pressure. 
These students can recognize a wide range of physical and chemical properties of matter and 
some of their interactions and understand some of the properties of light and sound. Also, 
they can infer relationships between structure and function. These students know the 
difference between plant and animal cells and can apply their knowledge of food as a source 
of energy to a practical situation. In addition, they are able to explain the impact of human 
activities on the environment and the economy. 



SOURCE: National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of Educational Progress. 
Washington, DC: Author. 
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Figure 1 .6 NAEP Science Achievement Levels 

Grade 12 




Basle 

(146) 



Prenehttt 

(178) 



Students performing at the Basic level demonstrate some knowledge and certain 
reasoning abilities required for understanding of the Earth, physical, and life sciences at 
a level appropriate to grade 12. In addition, they demonstrate knowledge of the themes of 
science (models, systems, and patterns of change) required for understanding the most 
basic relationships among the Earth, physical, and life sciences. They are able to conduct 
investigations, critique the design of investigations, and demonstrate a rudimentary 
understanding of the scientific principles. 

Twelfth-grade students performing at the Basic level are able to select and use appropriate 
simple laboratory equipment and write down simple procedures that others can follow. They 
also have a developmental ability to design complex experiments. These students are able to 
make classifications based on definitions such as physical properties and characteristics. 

Students at this level demonstrate a rudimentary understanding of basic models and can also 
identify some parts of physical and biological systems. They are also able to identify some 
patterns in nature and rates of change over time. These students have the ability to identify 
basic scientific facts and terminology and have a rudimentary understanding of the scientific 
principles underlying such phenomena as volcanic activity, disease transmission, and energy 
transformation. In addition, they have some familiarity with the application of technology. 

Students performing at the Proficient\eve\ demonstrate the knowiedge and reasoning 
abiiities required for understanding of the Earth, physicai, and iife sciences at a ievei 
appropriate to grade 12. in addition, they demonstrate knowiedge of the themes of 
science (modeis, systems, and patterns of change) required for understanding how these 
themes iiiustrate essentiai reiationships among the Earth, physicai, and iife sciences. 

They are abie to anaiyze data and apply scientific principies to everyday situations. 

Twelfth-grade students performing at the Proficient level are able to demonstrate a working 
ability to design and conduct scientific investigations. They are able to analyze data in 
various forms and utilize information to provide explanations and to draw reasonable 
conclusions. 

Students at this level have a developmental understanding of both physical and conceptual 
models and are able to compare various models. They recognize some inputs and outputs, 
causes and effects, and interactions of a system. In addition, they can correlate structure to 
function for the parts of a system that they can identify. These students also recognize that 
rate of change depends on initial conditions and other factors. They are able to apply 
scientific concepts and principles to practical applications and solutions for problems in the 
real world and show developmental understanding of technology, its uses, and its applications. 
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Figure 1.6 

Grade 12 
(continued) 


NAEP Science Achievement Levels 


I 

Advaaced \ 


Students performing at the Advanced \eye\ demonstrate the knowledge and reasoning 


(2i0) 


abiiities required for a solid understanding of the Earth, physical, and life sciences at a 
level appropriate to grade 1 2. In addition, they demonstrate knowledge of the themes of 
science (models, systems, and patterns of change) required for integrating knowledge 
and understanding of scientific principles from Earth, physical, and life sciences. 
Students can design investigations that answer questions about real-world situations and 
use their reasoning abilities to make predictions. 


i 


Twelfth-grade students performing at the Advanced level are able to design scientific 
investigations to solve complex, real-world situations. They can integrate, interpolate, and 
extrapolate information embedded in data to draw well-formulated explanations and 
conclusions. They are also able to use complex reasoning skills to apply scientific knowledge 
to make predictions based on conditions, variables, and interactions. 


! 

i 

i 

1 


Students at this level recognize the inherent strengths and limitations of models and can 
revise models based on additional information. They are able to recognize cause-and-effect 
relationships within systems and can utilize this knowledge to make reasonable predictions of 
future events. These students are able to recognize that patterns can be constant, exponen- 
tial, or irregular and can apply this recognition to make predictions. They can also design a 


V 


technological solution for a given problem. 



SOURCE: National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of Educational Progress. 
Washington, DC: Author. 



The Developmental Status 
of Achievement Levels 

The 1994 NAEP reauthorization law 
requires that the achievement levels be 
used on a developmental basis until the 
Commissioner of Education Statistics 
determines that the achievement levels are 
‘reasonable, valid, and informative to the 
pubhc.” Until the determination is made, 
the law requires the Commissioner and 
NAGB to state clearly the developmental 
status of the achievement levels in all 
NAEP reports. 



In 1993, the first of several congression- 
aUy mandated evaluations of the achieve- 
ment-level-setting process concluded that 
the procedures used to set the achievement 
levels were flawed and that the percentage 
of students at or above any particular 
achievement level cutpoint may be under- 
estimated.^^ Others have asserted that the 
weight of the empirical evidence does not 
support such conclusions.^^ 
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Improving America s Schools Act of 1994 (20 USC 9010) requires that the Commissioner base his determination 
on a congressionaUy mandated evaluation by one or more nationally recognized evaluation organizations. 

United States General Accounting Office. (1993). Education achievement standards: NAGB's approach yields misleading 
interpretations.U.S. General Accounting Office Report to Congressional Requestors. Washington, DC: Author. 

National Academy of Education. (1993). Setting perjormance standards for achievement: A report of the National Academy 
of Education panel on the evaluations of the NAEP trial state assessment: An evaluation of the 1992 achievement levels. 
Stanford, CA: Author. 

Cizek, G (1993). Reactions to National Academy of Education report Washington, DC: National Assessment Governing 
Board. 

Kane, M. (1993). Comments on the NAE evaluation of the NAGB achievement levels. Washington, DC: National 
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The most recent congressionally man- 
dated evaluation conducted by the Na- 
tional Academy of Sciences (NAS) relied 
on prior studies of achievement levels, 
rather than carrying out new evaluations, 
on the grounds that the process has not 
changed substantially since the initial 
problems were identified. Instead, the NAS 
Panel studied the development of the 1996 
science achievement levels. The NAS 
Panel basically concurred with earlier 
congressionally mandated studies. The 
Panel concluded that“NAEP’s current 
achievement level setting procedures 
remain fundamentally flawed. The judg- 
ment tasks are difiicult and confusing; 
raters’ judgments of different item types are 
internally inconsistent; appropriate validity 
evidence for the cut scores is lacking; and 
the process has produced unreasonable 
results.” 

A proven alternative to the current 
process has not yet been identified. The 
Deputy Commissioner of Education 
Statistics and the Board continue to call on 
the research community to assist in finding 
ways to improve standard setting for re- 
porting NAEP results. The NAS Panel 
accepted the continuing use of achieve- 
ment levels in reporting NAEP results, 
until such time as better procedures can be 
developed. Specifically, the NAS Panel 
concluded that “...tracking changes in the 



percentages of students performing at or 
above those cut scores (or, in fact, any 
selected cut scores) can be of use in de- 
scribing changes in student performance 
overtime.”’^ The National Assessment 
Governing Board urges all who are con- 
cerned about student performance levels to 
recognize that the use of these achievement 
levels is a developing process and is subject 
to various interpretations. The Board and 
the Deputy Commissioner beheve that the 
achievement levels are useful for reporting 
trends in the educational achievement of 
students in the United States. However, 
based on the congressionally mandated 
evaluations so far, the Deputy Commis- 
sioner agrees with the National Academy’s 
recommendation that caution needs to be 
exercised in the use of the current achieve- 
ment levels. Therefore, the Deputy Com- 
missioner concludes that these achievement 
levels should continue to be considered 
developmental and continue to be inter- 
preted and used with caution. 



Pellegrino, J.W., Jones, L.R., & Mitchell, K.J. (Eds.). (1999). Grading the nation’s report card: Evaluating NAEP and 
transforming the assessment of educational progress. Committee on the Evaluation of National Assessments of Educa- 
tional Progress, Board on Testing and Assessment, Commission on Behavioral and Social Sciences and Education, 
National Research Council, (pp. 182). Washington, DC; National Academy Press. 

Ibid., 176. 
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Sample Assessment Questions 

Three blocks of questions at each grade 
were released to the public following the 
administration of the NAEP 2000 science 
assessment and three blocks at each grade 
were released following the 1996 adminis- 
tration. All these questions can be found on 
the NAEP web site.^® The questions 
released from the 2000 assessment were 
also administered in the 1996 science 
assessment. Results for nine of the released 
questions, three from each of grades 4, 8, 
and 12, are presented in tables 1.1 through 
1.9. These questions illustrate the types of 
questions included in the assessment. 



The first three sample questions were 
administered at grade 4. Sample question 1 
is a hfe science question that asked students 
to recognize the function of the esophagus. 
Sample question 2 is an Earth science 
question that asked students to explain why 
the Earth never runs out of rain. Sample 
question 3 is a physical science question 
that required students to explain how 
they could find out which of the three 
differently shaped bottles would hold the 
most water. 



20 
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Grade 4 Sample Question 1: 




Look at the picture above, which shows some of the organs that can be found 
inside the human body. What is the main job of the organ labeled 1 ? 

® Carrying air 

• Carrying food 

CD Carrying blood 

CD Carrying messages from the brain 



Table 1.1 Sample Question 1 Results (Multiple-Choice) 



Overall percentage correct and percentages correct within each achievement-level range: 2000 



Grade 4 



Percentage correct within 
achievement-level intervals 



Overall percentage Basic 
correct^ 138-169* 



Proficient Advanced 

170-204* 205 and above* 



55 



55 75 90 



^Includes fourth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Grade 4 Sample Question 2: 



Think about where rain comes from and explain why the Earth never 
runs out of rain. 



Responses to this question were scored according to a three-level rubric as 
Unsatisfactory/lncorrect, Partial, or Complete. 



Sample “Complete” Response: 

« 

gg-f ra/n (f 

f=>.YrL ^ ra\/^5 K )' 



EeyiAj rf^irO^ (y<^ Hoju/v 

rrrjfyi/\y Jkjtejj^j pAJtUiA ^lcitx^ 



Table 1.2 Sample Question 2 Results (Short Constructed-Response) 



Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 
2000 



Overall percentage Basic Proficient Advanced 

“Complete"* 138-169* 170-204* 205 and above* 

28 26 45 65 

includes fourth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 



Grade 4 



Percentage correct within 
achievement-level intervals 
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Grade 4 Sample Question 3: 



You are going to the park on a hot day and need to take some water 
with you. You have three different bottles, as shown in the picture 
below. You want to choose the bottle that will hold the most water. 
Explain how you can find out which bottle holds the most water. 




Responses to this question were scored according to a three-level rubric as 
Unsatisfactory/Incorrect, Partial, or Complete. 



Sample “Complete" Response: 




cam ^2^ (X 



i/vk- 






— Oun^ Jiru9\ .QAieny 



CUncL. 







(yy\JL 









/ 






i 



M /Tn/rtL 






Cam. 



Table 1.3 Sample Question 3 Results (Short Constructed-Response) 



Overall percentage “Complete” and percentages “Complete” within each achievement-level range- 
2000 



Percentage correct within 
achievement-level intervals 



Overall percentage Basic Proficient Advanced 

"Complete” ♦ 138-169* 17li-204* 205 and above* 

5 4 10 23 

'Includes fourth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 



BESTCOPY AVAILABLE 

on 








The next three sample questions were 
administered at grade 8. Sample question 4 
is a physical science question that asked 
students to recognize that a magnet would 
not be helpful in separating a mixture of 
sand and salt. Sample question 5 assessed 
the domain of Earth science. It required 
students to state what they thought caused 



a monument to crumble and how to 
prevent further damage to the stone. 

Sample question 6 asked students to place 
8 animals into two groups based on a 
physical characteristic. They were also asked 
to name a second physical characteristic 
they could have used. This was a life sci- 
ence question. 



Grade 8 Sample Question 4: 



All of the following would be helpful in separating a mixture of sand 
and salt EXCEPT 

• a magnet 

CD a glass cup 

® a filter paper and funnel 

<D water 



Table 1.4 Sample Question 4 Results (Multiple-Choice) 



OveraU percentage correct and percentages correct within each achievement-level range: 2000 



Grades 



Percentage correct within 
achievement-level intervals 



Overall percentage Basic Proficient Advanced 

correct* 143-169* 170-207* 208 and above* 

59 59 71 81 

^Includes eighth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Grade 8 Sample Question 5: 



Cleopatra's Needle is a large stone monument that stood in an Egyptian 
desert for thousands of years. Then it was moved to New York City's 
Central Park. After only a few years, its surface began crumbling. 

What probably caused this crumbling? 



New York City wants to keep Cleopatra's Needle in the same location 
in Central Park. How can the city prevent further damage to the stone? 



Responses to this question were scored according to a three-level rubric as 
Unsatisfactory/Incorrect, Partial, or Complete. 

Sample “Complete” Response: 

What probably caused this crumbling? 









New York City wants to keep Cleopatra's Needle in the same location 
i n Cen tral Park. How can the city prevent further damage to the stone? 

^ 



A1 
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Table 1 .5 Sample Question 5 Results (Short Constructed-Response) 



Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 
2000 



Overall percentage Basic Proficient Advanced 

“Complete"^ 143-169* 170-207* 208 and above* 

28 28 47 71 

Includes eighth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 



Grade 8 



Percentage correct within 
achievement-level intervals 



Grade 8 Sample Question 6: 



Classify each of the eight living things listed below into one of two 
groups according to an important physical characteristic. 

Gorilla 

Parrot 

Snake 

Earthworm 

Jellyfish 

Sponge 

Fish 

Fly 

Group 1 Group 2 

What physical characteristic did you use in your classification? 



Name a different physical characteristic that you could have used. 
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Responses to this question were scored according to a four-level rubric as 
Unsatisfactory/Incorrect, Partial, Essential, or Complete. 



Sample “Complete” Response: 

Classify each of the eight living things listed below into one of two 
groups according to an important physical characteristic. 

Snake ^ 

Sponge 

Up Fish 

Fly 

group 1 Group 2 

What physical characteristic did you use in your classification? 



- cP- 

^'C o J^ aA. — 6 ^ 



Name a different physical characteristic that you could have used. 

Qyyx,ch /Y\j(f^ 



Table 1.6 Sample Question 6 Results (Extended Constructed-Response) 



Overall percentage Essential and percentages “Essential” within each achievement-level range: 2000 



Percentage “Essential" or better within 
achievement-level intervals 



Overall percentage Basic Proficient Advanced 

"Essential” or better* 143-169* 170-207* 208 and above* 

24 23 40 67 

'Includes eighth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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The last three sample questions were 
administered at grade 12. The earth science 
question shown in sample 7 required 
students to decide which of four statements 
most hkely explained the observation that 
the Sun appears to be slightly larger in 
January than in July. Sample question 8 
required students to describe a procedure 
for determining the density of a ring. It 
was classified as a physical science question. 
Sample 9, a life science question, was one 



of a set of 8 questions that probed students’ 
understanding of genetics and genetic 
engineering. Students were first asked to 
read an article about the use of viruses in 
genetic engineering and then asked to use 
the information in the article plus their 
own knowledge of genetics to answer the 
series of questions. The first question in the 
set asked students to state what a gene is, 
what it is made of, and its function. 



Grade 12 Sample Question 7; 



As observed with special instruments from Earth, the Sun appears in 
the sky to be slightly larger in January than in July. Which of the 
following accounts for this observation? 

• The Earth moves in an orbit that is not circular but is closer to 
the Sun in January than in July. 

CD The diameter of the Earth is not constant, but bulges slightly at 
the Equator and contracts slightly during the winter. 

CD The Earth's orbit is not in the same plane as the orbits of the 
other planets. 

CD The axis of rotation of the Earth is not perpendicular to the plane 
of its orbit but instead is tilted at an angle. 



Table 1.7 Sample Question 7 Results (Multiple-Choice) 



Overall percentage correct and percentages correct within each achievement-level range: 2000 



Grade 12 



Percentage correct within 
achievement-level intervals 



Overall percentage Basic Proficient Advanced 

correct* 146-177* 178-209* 210 and above* 



41 43 60 75 

^Includes twelfth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Grade 12 Sample Question 8: 



One characteristic that can be used to identify pure metals is density. 
If you determine the density of a pure metal, you can determine what 
the metal is, as shown in the table below. 



Metal 


Gold 


Lead 


Silver 


Copper 


Tin 


Density 

(gram/cm3) 


19.3 


11.3 


10.5 


8.9 


7.3 



Suppose that you have been given a ring and want to determine if it is 
made of pure gold. Design a procedure for determining the density of 
the ring. Explain the steps you would follow, including the equipment 
that you would use, and how you would use this equipment to deter- 
mine the ring's density. 



Responses to this question were scored according to a four-level rubric as 
Unsatisfactory/Incorrect, Partial, Essential, or Complete. 



Sample “Complete" Response: 

TI" , . . f - 
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Sample “Essential" Response: 



T '-tJoold uyf/qlo rf n 5c/ilf lYi 
i^yg vY'g,. ~X uTO;\d cdSfl — in' 
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Table 1.8 Sample Question 8 Results (Extended Constructed-Response) 



Overall percentage “Essential” or better and percentages “Essential” or better within each achievement- 
level range: 2000 



Grade 12 



Percentage ‘'Essential” or better within 
achievement-level intervals 



Overall percentage Basic Proficient Advanced 

“Essential” or better^ 146-177* 178-209* 210andabove* 



19 



18 



58 



89 



includes twelfth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Grade 12 Sample Question 9: 



Based on your knowledge of genetics and the information in the 
preceding passage, answer the following questions. 

What is a gene? What is it made of? What is the major function 
of a gene? 



Responses to this question were scored according to a four-level rubric as 
Unsatisfactory/Incorrect, Partial, Essential, or Complete. 

Sample “Complete” Response: 



What is a gene? What is it made of? What is the major function 
of a gene? 



(H . dn AMxscl m/A.^ ^ 






Sample “Essential” Response: 



What is a gene? What is it made of? What is the major function 
of a gene? 

— 0<A/c* 

— qyyydL QAI^ 
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Table 1.9 Sample Question 9 Results (Extended Constructed-Response) 



Overall percentage “Essential” or better and percentages “Essential” or better within each achievement- 
level range: 2000 



Overall percentage 


Basic 


Proficient 


Aifvanced 


“Essential'' or better ^ 


146-177* 


178-209* 


210 and above* 


22 


24 


44 


56 



♦Includes twelfth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Maps of Selected 
Item Descriptions 

The science performance of fourth-, 
eighth-, and twelfth-graders can be illus- 
trated by maps that position questions from 
the assessment onto the O-to-300 scale. The 
resulting item maps are visual representa- 
tions of how the difficulty of each question 
compares with a student’s performance on 
the entire test.^^ The descriptions used on 
these maps focus on the science knowledge 
or s kill needed to answer the question. For 
multiple-choice questions, the description 
indicates the knowledge or skill demon- 
strated by selection of the correct option; 
for constructed-response questions, the 
description takes into account the knowl- 
edge or skill specified by the different levels 
of scoring criteria for that question. Seven 
of the questions described on the item 
maps are included among the sample 
questions in the preceding section. Each of 
these sample questions is identified as such 
on the item map. 

Figures 1.7 through 1.9 are item maps 
for grades 4, 8, and 12, respectively. For 
each question indicated on the map, stu- 
dents who scored above the scale point had 
a higher probability of successfully answer- 



ing the question, and students who scored 
below the scale point had a lower probabil- 
ity of successfully answering the questions. 
The map location for each question identi- 
fies where that question was answered 
successfully by at least 65 percent of stu- 
dents for constructed-response questions 
and at least 74 percent of students for a 
four-option multiple-choice question. 

As an example of how to interpret the 
item maps, consider the multiple-choice 
question in figure 1.7 that maps at score 
point 188. Fourth-graders were required to 
identify the function of a labeled human 
organ. Students who scored at or above 
188 on the NAEP scale had a 74 percent 
chance of answering this question correctly. 
Students who scored below 188 had less 
than a 74 percent chance of doing so. This 
does not mean that all students scoring 188 
or above always answered the question 
correctly, or that students scoring below 
188 always answered the question incor- 
rectly. Rather, the item map indicates a 
higher or lower probabifity of answering 
the question successfully depending on 
students’ overall science knowledge and 
s kill s as measured by the NAEP scale. 



Details on the procedures used to develop item maps are provided in appendix A, pp. 174—175. 
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As another example of how to interpret 
the item maps, consider the question in 
figure 1.8 that maps at score point 194. 
Eighth-graders were asked to classify eight 
different organisms into two groups based 
on a physical characteristic. They were also 
asked to name a second physical character- 
istic they could have used. Students’ re- 
sponses to this constructed-response 
question were rated according to a four- 
level scoring guide that distinguished 
between “Unsatisfactory,” “Partial,” “Essen- 
tial,” and “Complete.” As with all con- 
structed-response questions portrayed on 



the item maps, the description of this item 
takes into account the requirements for a 
response to be rated at a certain level 
according to the scoring criteria for that 
question. With this question, the descrip- 
tion is based on the level of performance 
required for a score of “Essential” or better. 
Students who scored at or above 194 on 
the NAEP scale had at least a 65 percent 
chance of demonstrating the knowledge 
and skill required to receive a rating of 
“Essential” or better on this question. 
Students who scored below 194 had less 
than a 65 percent chance of doing so. 
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NAEP Science Scale 







uu 

h 



Map of selected item 
descriptions on the 
National Assessment 
of Educational 



science scale for 
grade 4 



260 

250 

240 

230 

220 



256 Predict and explain the effect of fertilizer on algae growth in pond 
250 Recognize source of stored energy in beans 



242 Draw part of the food web of an ecosystem 

235 Predict and explain cause of temperature increase in buckets of water left in sunlight 



227 State and explain features of houses that combat heat or cold from weather 
226 Explain roughly how to determine which of three bottles holds the most water — 

Sample Question 3 



This map describes 
the knowledge or 



Advanced 



210 



200 



with answering 
individual science 
questions. The map 
identifies the score 
point at which 

gh 



190 



195 Recognize likelihood of rain based on type of cloud 



180 



188 Identify function of labeled human organ — Sample Question 1 

185 Predict effect of one magnet on another 

182 Identify which animals produce very large number of offspring 



Proficient 



170 , 



176 Recognize safest place to go in case one hears thunder and explain why 
175 Explain how bicycle reflectors work 



160 



’ 1^7' Explain the' good ahd bad points of sorhe'given rfiethods of garbage disposal 
162 Identify producer in given ecosystem 
160 Identify forms of precipitation 
160 Recognize cause of window rattling due to thunder 



150 



151 Recognize vibrations being common to several actions 



Basic 



140 



140 Name and explain some of the survival features of deer’s body 



130 



136 Recognize best information for identifying a rock 



120 



110 



100 



101 Recognize instrument used to see stars 



90 

-G- 



93 Name one of the predators in given ecosystem 



ERIC 



NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. 

* Each grade 4 science question in the 2000 assessment was mapped onto the NAEP 0-300 science scale. The position of the question on the scale represents the scale score attained by 
students who had a 65 percent probability of successfully answering a constructed-response question or a 74 percent probability of correctly answering a four-option multiple-choice 
question. Only selected questions from among those that were released after the 2000 assessment are presented. Scale score ranges for science achievement levels are referenced on the map. 
To interpret the item map, consider, for example, the multiple-choice question that maps at a scale score of 250 for grade 4. This question concerns the source of stored energy in beans. 
Mapping the question at the 250 scale score indicates that at least 74 percent of the students performing at this point answered the question correctly. Among students with lower 
scores, less than 74 percent answered this question correctly. 

SOURCE: National Center for Education Statistics. National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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NAEP Science Scale 
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jyu 

260_ 

25Q 



_251 Explain cause of echo in auditorium 



240. 



Map of selected item 
descriptions on the 
National Assessment 
of Educational 
Progress 
science scale for 
grade 8 

This map describes 
the knowledge or 
skills associated 
with answering 
individual science 
questions. The map 
Identifies the score 
point at which 
students had a high 
probability of 
successfully 
answering the 



230 



241 Describe part of test to investigate behavior of paramecia 

237 Recognize reason for controlling air pressure in airplane 

235 State that length of shadow at noon is different in summer and winter 

233 Identify top carnivore in pond ecosystem 



220 

Advanced 2^TQ_ 

20(1 



204 

203 

197 

194 



187 

184 



Proficient 




190 

lao 

m, 

ifid 
150. 
14d 
130 

m 

m 



Recognize tools useful for separating sand and salt mixture—Samp\e Question 4 
Recognize appliance that converts energy to mechanical work 

Recognize location of Moon with respect to Sun and Earth 

Classify living organisms based on physical characteristic — Sample Question 6 

Identify direction of river flow on contour map 

Identify place in human body where digestion of protein occurs 



166 Explain cause or effect of competition in pond ecosystem 



160 

156 

155 

155 



Describe either production mechanism or method of travel of sound 
Identify why water is most important to living organisms 
Identify effect of add rain on pond ecosystem 
Explain that length of shadow relates to position of Sun in the sky 



144 Explain/easpn^or^ prevention methods of rnonumept^crurnblinjg-;-Sample Question 5 



137 

133 



121 

117 



Explain part of procedure of measuring change in length of shadow throughout the day 
Discuss a factor to be considered in deciding to use ethyl alcohol as automobile fuel 



Name some things that animals get from food for survival 

Give partial explanation of how color of clothing relates to coolness on sunny day 



111 Identify organisms most likely to live in tropical rain forest 



100 . 

‘i 

-e- 
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NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. 

* Each grade 8 science question In the 2000 assessment was mapped onto the NAEP 0-300 science scale. The position of the question on the scale represents the scale score attained by 
students who had a 65 percent probability of successfully answering a constructed-response question or a 74 percent probability of correctly answering a four-option multiple-choice 
question. Only selected questions from among those that were released after the 2000 assessment are presented. Scale score ranges for science achievement levels are referenced on the map. 

To interpret the item map, consider, for example, the constructed-response question that maps at a scale score of 194 at grade 8. This question concerns the classification of living 
organisms. Scoring of responses to this question allowed for partial credit by using a four-level scoring guide. Mapping the question at the 194 scale score indicates that at least 65 percent 
of the students performing at this point achieved a score of 2 (“Partial") on the question. Among students with lower scores, less than 65 percent received a score of 2 on the question. 
SOURCE: National Center for Education Statistics. National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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NAEP Science Scale 
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Grade 12 
item Map 



25a 



252 Identify relationships from given evolutionary tree 



Z4Q_ 



Map of selected item 
descriptions on the 
National Assessment 
of Educational 
Progress 
science scale for 
grade 12 



23a 



232 Predict volume of 0- given parameters 



??n - 



223 Explain why Sun appears larger in January than in July— Sample Question 7 



Advanced 



219 Name plants present when first amphibians appeared 
213 Predict length of year on Mars 



210 212 Estimate age of wood using radioactive dating 



This map describes 
the knowledge or 
skills associated 
with answering 
individual science 
questions. The map 
identifies the score 
point at which 
students had a high 
probability of 



2 QQ_ 






199 Explain cause of convection in atmosphere 

194 Make determination about composition of ring based on its density — Sample Question 8 



Proficient 



18Q 



187 Discuss cause or effect of genetic mutation 
186 Name a disadvantage of using recombinant DNA technology 
Explain how mountain forms near continental plate boundary 
Identify source of energy released in nuclear decay 



185 

181 



m 




lea 



169 Describe similarities and differences between models of atom and solar system 
166 Name difference between offspring of sexually and asexually reproducing animals 
163 Draw simplified model of solar system 



Basic 150 



153 Predict distance of new planet from given and derived information 



L4D 



133_ 



12a 



126 Plot period v. distance from Sun for planets given data table 

122 Name a way that the real solar system is different from the model drawn by the student 
121 State cause and effect of AIDS 



iin . 



10 Identify planet with longest year from data table 



inn 






m 



80 Describe some modes of AIDS transmission 



‘i 



a 
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NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. 

* Each grade 12 science question in the 2000 assessment was mapped onto the NAEP 0-300 science scale. The position of the question on the scale represents the scale score attained by 
students who had a 65 percent probability of successfully answering a constructed-response question or a 74 percent probability of correctly answering a four-option multiple-choice question 
Only selected questions from among those that were released after the 2000 assessment are presented. Scale score ranges for science achievement levels are referenced on the map. 

To interpret the item map, consider, for example, the constructed-response question that maps at a scale score of 163 at grade 12. This question asked students to draw a simplified model of 
the solar system. Scoring of responses to this question allows for partial credit by using a three-level scoring guide. Mapping the question at the 163 scale score indicates that at least 65 
percent of the students performing at this point achieved a score of 3 ("Complete") on the question. Among students with lower scores, less than 65 percent received a score of 3 on the question. 
SOURCE: National Center for Education Statistics. National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Interpreting NAEP Results 

The average scores and percentages pre- 
sented in this report are based on represen- 
tative samples of students rather than on 
the entire population of students. More- 
over, the collection of questions used at 
each grade level is but a sample of the 
many questions that could have been asked 
that measure the content and skills outlined 
in the NAEP science framework. As such, 
the results are subject to a measure of 
uncertainty, reflected in the standard error 
of the estimates. The standard errors for the 
estimated scale scores and percentages in 
this report are provided in appendix B. 

The differences between scale scores and 
between percentages discussed in the 
following chapters take into account the 
standard errors associated with the esti- 
mates. Comparisons are based on statistical 
tests that consider both the magnitude of 
the difference between the group average 
scores or percentages and the standard 



errors of those statistics. Throughout this 
report, differences between scores and 
between percentages are pointed out only 
when they are significant from a statistical 
perspective. All differences reported are 
significant at the 0.05 level with appropri- 
ate adjustments for multiple comparisons. 
The term “significant” is not intended to 
imply a judgment about the absolute 
magnitude of the educational relevance of 
the differences. It is intended to identify 
statistically dependable population differ- 
ences to help inform dialogue among 
policymakers, educators, and the public. 

Readers are cautioned against interpret- 
ing NAEP results in a causal sense. Infer- 
ences related to subgroup performance or 
to the effectiveness of public and nonpubhc 
schools, for example, should take into 
consideration the many socioeconomic and 
educational factors that may also impact on 
science performance. 
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Overview of Remaining Chapters 

The results in chapters 2 and 3 of the 
report are based on a set of data with no 
accommodations offered to students. 
Findings are presented for the nation, for 
regions, for participating jurisdictions, and 
for the major reporting subgroups included 
in all NAEP report cards. Changes since 
the 1996 assessment are noted where the 
data permit comparisons. State-by-state 
results are included for the states and 
jurisdictions that participated in the science 
assessment at grades 4 and 8. Chapter 4 
presents an overview of the second set of 
results — those that include students who 
were provided accommodations during the 
test administration. By including these 
results in the nation s science report card, 
the NAEP program continues a phased 
transition toward a more inclusive report- 
ing sample. Future assessment results will 
be based solely on a student and school 
sample in which accommodations are 
permitted. 



Chapter 5, which is based on the data 
with no accommodations offered, looks at 
factors that may influence teaching and 
learning, such as teacher certification and 
classroom practices. It includes information 
on the types of science courses students 
were taking at the time of the assessment. 

This report also contains appendices that 
support or augment the results presented. 
Appendix A contains an overview of the 
NAEP science framework and specifica- 
tions, information on the national and state 
samples, and a more detailed description of 
the major reporting subgroups featured in 
chapters 2 and 3. Appendix B contains the 
fill] data with standard errors for aU tables 
and figures in this report. Appendix C 
presents selected state-level contextual 
variables from non-NAEP sources that may 
be associated with student performance. 
Appendix D contains a list of the NAEP 
science committee members. 

Detailed information about the mea- 
surement methodology and data analysis 
techniques is available in the NAEP 2000 
Technical Report. 
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Average Scale Score and Achievement- 
Level Results for the Nation and States 



The extent to which the nation is realizing one of the goals 
set at the National Education Summit in 1989 — to ensure 
that students leaving the fourth, eighth, and twelfth grades 
demonstrate competency in core subjects — can now be 
examined in hght of results obtained from two 
administrations of the National Assessment of 
Educational Progress (NAEP) science assessment.’ 
The results of the science assessment administered in 
1996 showed 29 percent of fourth- and eighth- 
graders, and 21 percent of twelfth-graders 
demonstrated competency over challenging subject 
matter, including subject-matter knowledge, 
application of such knowledge to real-world 
situations, and analytical skills appropriate to the 
subject matter.^ Given the extensive push within the 
United States in the past decade to reform science 
teaching and learning, there is an interest in 
determining if the results of the NAEP 2000 science 
assessment, compared to the results from 1996, would 
positively reflect these reforms by showing an increase in the 
percentage of students demonstrating competency over 
challenging material. 

This chapter presents the NAEP 2000 science results for 
the nation at grades 4, 8, and 12 and for participating states 
and jurisdictions at grades 4 and 8. Student performance on 
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^ U.S. Department of Education. (1991). America 2000: An education Washington, 

DC: Author. 

^ Bourque, M. L., Champagne, A. B., & Crissman, $. (1997). 1996 science performance 
standards: Achievement results for the nation and the states. Washington, DC: National 
Assessment Governing Board. 
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NAEP is reported in two ways: one, as 
average scores on the NAEP science 
composite scale, which ranges from 0 to 
300, and two, as percentages of students 
who attained each of the three science 
achievement levels: Basic, Proficient, and 
Advanced. Discussion of students’ progress 
over time is based on a comparison of the 
results in 2000 to those from the 1996 
assessment. This comparison is possible 
because the assessments shared a common 
set of science questions based on the 
current science framework and because the 
populations of students were sampled and 
assessed using comparable procedures. 

Readers are reminded that differences 
between scale scores and percentages 
discussed in this chapter take into account 
the standard errors associated with the 
estimates. Thus, a small difference between 
scores in one comparison may be signifi- 
cant while a similar or larger difference 
between scores in another comparison may 
not be statistically significant. 



The results presented in this chapter are 
based on a representative sample of stu- 
dents assessed under conditions that did not 
permit accommodations for special-needs 
students. These were the same conditions 
under which the 1996 science assessment 
was administered, thus making it possible to 
report changes in student performance 
across the assessment years. A second set of 
results that reflect part of a phased transi- 
tion toward a more inclusive reporting 
sample in which accommodations were 
permitted is presented in chapter 4. 

National Scale Score Results 

Figure 2.1 presents the average science 
scale scores of fourth-, eighth-, and 
twelfth-grade students attending both 
public and nonpublic schools in 1996 and 
2000. There were no statistically significant 
differences observed in average science 
scores from 1996 to 2000 at grades 4 and 8, 
and a decrease at grade 12 from an average 
score of 150 to 147. 



Figure 2.1 National average science scale scores, grades 4, 8, and 12: 1996 and 2000 

National Scale Score 
Results 




★ Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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National Percentile Score Results 

Changes in student performance can also 
be examined by looking at the percentile 
scores on the NAEP science scale across 
assessment years. The advantage of looking 
at data in this way is that it shows whether 
changes in the national average scores are 
reflected in scores across the performance 
distribution. 

Figure 2.2 shows the science scores for 
grades 4, 8, and 12 at the 10*, 25*, 50*, 
75*, and 90* percentiles in both 1996 and 
2000. At grade 4, there was no significant 



difference observed in the percentile scores 
since 1996. Although there was no signifi- 
cant difference observed in the national 
average score at grade 8 between 1996 and 
2000, there was an increase in the scale 
score at the 90* percentile — from 192 in 
1996 to 195 in 2000 — indicating improve- 
ment for the highest-performing students. 
At grade 12, the score at the 50* percentile 
declined between 1996 and 2000, indicat- 
ing that the recent performance decline 
was primarily focused in the middle of the 
score distribution. 



Figure 2.2 National science scale score percentiles, grades 4, 8, and 12: 1996 and 2000 

National Performance 
Distribution 




90th 



75th 



50th 



25th 



10th 





★ Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



BEST COPY AVAILABLE 



o 

ERIC 



57 



CHAPTER 2 



SCIENCE REPORT CARD 



39 






















Achievement-Level Results 
for the Nation 

The science achievement levels — Basic, 
Proficient, and Advanced — used to report 
NAEP results were estabhshed by the 
National Assessment Governing Board 
(NAGB) in 1996. A discussion of the 
achievement-setting process can be found 
in chapter 1 of this report together with 
descriptions of what students in grades 4, 8, 
and 12 know and can do at each of the 
three achievement levels. 

Achievement-level results for the 
nation s fourth-, eighth-, and twelfth-grade 
students are presented in figure 2.3. Results 
are presented in two ways: as the percent- 
age of students within each achievement 
level interval, and as the percentage of 
students at or above the Basic and Proficient 
levels. It is necessary to keep in mind that 
the percentages at or above specific 
achievement levels are cumulative. There- 
fore, included among the percentage of 
students at or above the Basic level are also 
those who have achieved the Proficient and 
Advanced levels of performance, and in- 
cluded among students at or above the 
Proficient level are also those who have 
attained the Advanced level of performance. 



As shown in figure 2.3, performance at 
or above the Proficient level — the achieve- 
ment level identified by NAGB as the level 
that all students should reach — was attained 
by 29 percent of fourth-graders, 32 percent 
of eighth-graders, and 18 percent of 
twelfth-graders in 2000. 

No statistically significant differences 
were detected on the NAEP measure at 
grade 4 (29 percent were at or above the 
Proficient level in both 1996 and 2000). 
However, at the eighth-grade level, some 
progress as demonstrated on the NAEP 
measure has been made. More students 
demonstrated competency over challenging 
science material; 32 percent were at or 
above the Proficient level in 2000 compared 
to 29 percent in 1996. At grade 12, the 
percentage of students at or above Basic 
declined between 1996 and 2000, from 57 
percent to 53 percent. The apparent 
decline in the percentage of twelfth- 
graders at or above the Proficient level was 
not found to be statistically significant. 



er|c, 



lAPTER 2 



SCIENCE REPORT CARD 



58 



Figure 2.3 

National Achievement' 
Level Results 



Percentage of students within each science achievement-level range and at or above 
achievement levels, grades 4, 8, and 12: 1996 and 2000 



How to read these figures: 

- The italicized 
percentages to the 
right of the shaded 
bars represent the 
percentages of 
students at or above 
Basic Proficient 

- The percentages in 
the shaded bars 
represent the 
percentages of 
students within each 
achievement level. 
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★ Significantly different from 2000. 

NOTE: Percentages within each science achievement level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Results for Regions of the Country 

This section examines results for four 
regions of the country: Northeast, South- 
east, Central, and West. A listing of the states 
and other jurisdictions within these regions 
is provided in appendix A. Figure 2.4 
presents scale score results by region. At 
grades 4 and 8, there were no statistically 
significant differences in the performance 
of students attending schools in the North- 
east, Southeast, Central, and West regions 
between 1996 and 2000. At grade 12, 
however, the average science score for 
students attending schools in the Central 
region was lower in 2000 than in 1996. 



Comparisons between the regions in the 
2000 assessment show that fourth-grade 
students attending schools in the Northeast 
and Central regions outperformed their 
peers in the West and Southeast. In addi- 
tion, grade 4 students in the West had 
higher scores than students in the Southeast. 
Eighth-grade students attending schools in 
the Northeast and Central regions had 
higher average scores than their peers in the 
West and Southeast. Twelfth-grade students 
attending schools in the Southeast were 
outperformed by their peers in the North- 
east and Central regions. 



Figure 2.4 


National science scale score results by region of the country, grades 4 8 and 1 2- 


National Scale Score 


1996 and 2000 


Results by Region 





Grade 4 I Grade 8 I Grade 12 

'96 '00 I '96 '00 I '96 '00 




• Northeast a Central 

□ Southeast q 



* Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Figure 2.5 presents the achievement-level 
results by region. At grades 4 and 8, there 
were no statistically significant changes in 
the percentages of students at or above the 
Basic and Proficient levels between 1996 and 
2000 in any of the four regions. The one 
percentage point increase at the Advanced 
level in the Southeast at grade 8 was, 
however, statistically significant. At grade 
12, the percentage of students at or above 
the Basic and Proficient levels decreased in 
the Central region between 1996 and 2000. 

A number of differences can be seen 
when the results for each of the three 
grades in 2000 are compared between the 
regions. At grade 4, both the Northeast and 
Central regions had higher percentages of 
students at or above the Basic level than in 



the Southeast and West, and higher percent- 
ages at or above Proficient than in the 
Southeast. At grade 8, both the Northeast 
and the Central regions had higher per- 
centages of students at or above the Basic 
and Proficient levels than did the Southeast 
and West. In addition, the percentage of 
eighth-graders at or above the Basic level in 
the Central region was higher than the 
percentage of eighth-graders in the North- 
east. At grade 12, the Northeast and Central 
regions had higher percentages of students 
at or above the Basic ItvA than did the 
Southeast. There was no statistically signifi- 
cant difference between the regions in the 
percentage of students at or above the 
Proficient level at grade 12. 
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Figure 2.5a 



Percentage of students within each science achievement-level range and at or above 
National Achievement- achievement levels, by region of the country, grade 4: 1996 and 2000 
Level Results by 
Region 
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Figure 2.5b 

National Achievement- 
Level Results by 
Region (continued) 



Percentage of students v/ithin each science achievemenMevel range and at or above 
achievement levels, by region of the country, grade 8: 1996 and 2000 
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Figure 2.5c 



Percentage of students within each science achievement-level range and at or above 
National Achievement- achievement levels, by region of the country, grade 1 2: 1 996 and 2000 
Level Results by 
Region (continued) 
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NOTE: Percentages within each science achievement level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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state Results 

In addition to the national results, science 
performance data were also collected for 
students in grades 4 and 8 who attended 
public schools in states and other jurisdic- 
tions that chose to participate in the 
assessment. Although 45 states and jurisdic- 
tions participated at grade 4, and 44 states 
and jurisdictions participated at grade 8, 
not all met minimum school participation 
guidelines for reporting their results in 
2000. (See appendix A for details on 
participation and reporting guidelines.) 
Results from the 2000 assessment for 
grades 4 and 8 in Wisconsin and for grade 
8 in the Virgin Islands are not included in 
this report because they failed to meet the 
minimum public school participation rate 
of 70 percent. Jurisdictions that failed to 
meet one or more of the other participa- 
tion guidelines are noted in each of the 
tables. Results from both the 1996 and 
2000 state assessments are presented for 
grade 8, but results from 2000 only are 
reported at grade 4 since there was no 
state-level assessment administered to 
fourth-graders in 1996. Tables presenting 
state-level results at grade 8, indicate 
statistically significant changes across years 
when examining only one jurisdiction at a 
time (^), and when using a multiple com- 
parison procedure based on aU the jurisdic- 
tions that participated (t). Only those 
differences based on the multiple compari- 
son procedure are discussed. 

In examining the “accommodations- 
not-permitted” results for jurisdictions 
presented in this chapter, it should be noted 
that schools participating in the NAEP 
assessments under these conditions were 
permitted to exclude those students who 
could not be assessed meaningfully without 



accommodations. Exclusion rates vary 
across jurisdictions not only because of 
differences in the implementation of the 
Individuals with Disabilities Education Act 
(IDEA), but also because of population 
shifts in the percentage of students classi- 
fied with disabilities (SD) and, especially, 
limited EngHsh proficient (LEP) students. 
Therefore, comparisons of assessment 
results across jurisdictions and within 
jurisdictions across years should be made 
with caution. The percentage of students 
excluded from the assessment has implica- 
tions for the representativeness of the 
sample assessed within a jurisdiction. No 
adjustments have been made for differing 
exclusion rates across jurisdictions or across 
years. Thus, a comparison within a jurisdic- 
tion across years or between two jurisdic- 
tions may be based on samples with exclu- 
sion rates that differ considerably. The 
exclusion rates for each jurisdiction are 
presented in appendix A. 

Scale Score Results by 
Jurisdiction 

The average scale scores of public school 
students for participating jurisdictions are 
presented in table 2.1 for grade 4 and table 
2.2 for grade 8. Whereas the national 
results shown in previous sections of this 
chapter represent both pubhc and 
nonpublic schools combined, the national 
average scores shown in each of these tables 
represent the performance of pubhc school 
students only. Of the 36 jurisdictions that 
participated in both the 1996 and 2000 
state level assessments at grade 8, three 
showed score gains in 2000: Missouri and 
the Department of Defense domestic 
schools and overseas schools (DDESS and 
DoDDS). 
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Table 2.1 State Average Score Results, Grade 4 



Average science scale score 


results by state for grade 4 public schools: 2000 


Nation 


148 


Alabama 


143 


Arizona 


141 


Arkansas 


144 


California ^ 


131 


Connecticut 


156 


Georgia 


143 


Hawaii 


136 


Idaho ^ 


153 


Illinois ^ 


151 


Indiana ^ 


155 


Iowa ^ 


160 


Kentucky 


152 


Louisiana 


139 


Maine ^ 


161 


Mainland 


146 


Massachusetts 


162 


Michigan ^ 


154 


Minnesota ^ 


157 


Mississippi 


133 


Missouri 


156 


Montana ^ 


160 


Nebraska 


150 


Nevada 


142 


New Mexico 


138 


New York ^ 


149 


North Carolina 


148 


North Dakota 


160 


Ohiot 


154 


Oklahoma 


152 


Oregon ^ 


150 


Rhode Island 


148 


South Carolina 


141 


Tennessee 


147 


Texas 


147 


Utah 


155 


Vermont ^ 


159 


Virginia 


156 


West Virginia 


150 


Wyoming 


158 


Other Jurisdictions 


American Samoa 


51 


DDESS 


157 


DoDDS 


156 


Guam 


110 


Virgin Islands 


116 



t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: National results are based on the national sample, not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 



ERIC 

HAPTER 2 * 




SCIENCE REPORT CARD 




Table 2.2 State Average Score Results, Grade 8 



Average science scale score results by state for grade 8 public schools: 1996 and 2000 





1996 


2000 


Nation 


148 


149 


Alabama 


139 


141 


Arizona ^ 


145 


146 


Arkansas 


144 


143 


California ^ 


138* 


132 


Connecticut 


155 


154 


Georgia 


142 


144 


Hawaii 


135 


132 


Idaho ^ 


— 


159 


Illinois ^ 


— 


150 


Indiana ^ 


153 


156 


Kentucky 


147* 


152 


Louisiana 


132 


136 


Maine ^ 


163* 


160 


Maryland 


145 


149 


Massachusetts 


157 


161 


Michigan ^ 


153 


156 


Minnesota ^ 


159 


160 


Mississippi 


133 


134 


Missouri 


15P 


156 


Montana ^ 


162 


165 


Nebraska 


157 


157 


Nevada 


— 


143 


New Mexico 


141 


140 


New York ^ 


146 


149 


North Carolina 


147 


147 


North Dakota 


162 


161 


Ohio 


— 


161 


Oklahoma 


— 


149 


Oregon ^ 


155 


154 


Rhode Island 


149 


150 


South Carolina 


139 


142 


Tennessee 


143 


146 


Texas 


145 


144 


Utah 


156 


155 


Vermont ^ 


157* 


161 


Virginia 


149 


152 


West Virginia 


147 


150 


Wyoming 


158 


158 


Other Jurisdictions 


American Samoa 


— 


72 


DDESS 


153* 


159 


DoDDS 


155* 


159 


Guam 


120 


114 



* Significantly different from 2000 if only one jurisdiction or the nation is being examined. 

♦ Significantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on all jurisdictions that 
participated both years. 

1 Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 
NOTE: National results are based on the national sample, noton aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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The maps in figures 2.6 and 2.7 compare 
state and national average scores at grades 4 
and 8, respectively. At grade 4, 20 jurisdic- 
tions had scores that v/eve higher than the 
national average score, 13 had scores that 
were lovv^er than the national average, and 
no statistically significant differences Avere 
detected between the state and national 



average for 11 states. At grade 8, 18 juris- 
dictions had scores that were higher than 
the national average score, 13 had scores 
that were lower than the national average, 
and no significant differences were detected 
between the state and national average for 
11 states. 



Figure 2.6 


Comparison results of state and national average science scale scores 


State V. National 


for grade 4: 2000 


Scale Score, 




Grade 4 
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NM 














1 MS 






TX 







Samoa 
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state has higher average scale score than nation. 



r I state average score was not found to be 
significantly different from the nation. 



State has lower average scale score than nation. 






State did not meet the minimum participation 
rate guidelines (see technical notes). 

State did not participate in the NAEP 
2000 science state assessment. 



NOTE: Caution should be exercised when interpreting 
comparisons among states and other jurisdictions. 

NAEP performance estimates are not adjusted to 
account for the socioeconomic or demographic, 
differences among states and jurisdictions. 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Comparison results of state and national average science scale scores 
for grade 8: 2000 



Figure 2.7 

State V. National 
Scale Score, 
Grade 8 




State has higher average scale score than nation. 



I I State average score was not found to be 
significantly different from the nation. 

I I State has lower average scale score than nation. 






State did not meet the minimum participation 
rate guidelines (see technical notes). 

State did not participate in the NAEP 
2000 science state assessment. 



NOTE: Caution should be exercised when interpreting 
comparisons among states and other jurisdictions. 

NAEP performance estimates are not adjusted to 
account for the socioeconomic or demographic, 
differences among states and jurisdictions. 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Cross-State Scale Score 
Comparisons 

Figures 2.8 and 2.9 display the differences 
between the scale scores for all possible 
pairings of participating jurisdictions at 
grades 4 and 8, respectively. The variation 
in shading indicates whether a jurisdiction 
listed across the top of the figure had a 
score that was higher than, lower than, or 
not significantly different from other 
jurisdictions. Within each figure, jurisdic- 
tions are ranked from highest to lowest 
average scale score, both from left to right 
across the columns and down the rows. For 
example in figure 2.8, the first cell in the 
second row compares the average score at 
grade 4 in Massachusetts (MA) to the 
average score in Maine (ME). The lack of 
shading in this cell indicates that there was 



no statistically significant difference found 
between the scores in these two states. 
Moving down the first column to Wyo- 
ming (WY), the shading changes to indi- 
cate that the average score in Massachusetts 
was higher than that in Wyoming. At 
grade 4, the top 6 states had average scores 
that were not found to differ significantly 
from one another. These states were Iowa, 
Maine, Massachusetts, Montana, North 
Dakota, and Vermont. At grade 8, Idaho, 
Maine, Massachusetts, Minnesota, North 
Dakota, Ohio, Vermont, and the Depart- 
ment of Defense domestic schools and 
overseas schools all performed similarly 
(i.e., no significant differences were de- 
tected in the average scores of these 9 
jurisdictions) and were only outperformed 
by Montana. 
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Figure 2.8: Cross-State Scale Score Comparisons, Grade 4 



Comparisons of average science scale scores for grade 4 public schools: 2000 

Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction’s abbreviation to the key below to determine whether the average science scale score of this jurisdiction is higher than, the 
same as, or lower than the jurisdiction in the column heading. For example, in the column under Wyoming, Wyoming’s score was lower than Massachusetts 



and Maine, about the same as all the states from Iowa through Michigan, and higher than the remaining states down the i 
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Jurisdiction has statistically significantly higher average scale 
score than the jurisdiction listed at the top of the figure. 

No statistically significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statistically significantly lower average scale 
score than the jurisdiction listed at the top of the figure. 



BEST COPY AVAILABLE 



o 




The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Signihcance is determined by an 
application of a multiple comparison procedure (see appendix A). 

++lndicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Differences between states and jurisdictions may be partially 
explained by other factors not included in this figure. 

SOURCE; National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP). 2000 Science Assessment. 
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Figure 2.9; Cross-State Scale Score Comparisons, Grade 8 



Comparisons of average science scale scores for grade 8 public schools: 2000 

Instructions; Read dMQ the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction's abbreviation to the key below to determine whether the average science scale score of this jurisdiction is higher than, the 
same as, or lower than the jurisdiction in the column heading. For example, in the column under Indiana, Indiana's score was lower than Montana, 
Massachusetts, Vermont, and North Dakota, about the same as all the states from Ohio through Kentucky, and higher than the remaining states down 
the column. 
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□ 

□ 

□ 



Jurisdiction has statistically significantly higher average scale 
score than the jurisdiction listed at the top of the figure. 

No statistically significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statistically significantly lower average scale 
score than the jurisdiction listed at the top of the figure. 
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The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Significance is determined by an 
application of a multiple comparison procedure (see appendix A). 

++lndicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS; Department of Defense Dependents Schools (Overseas). 

NOTE: Differences between states and jurisdictions may be partially 
explained by other factors not included in this figure. 

SOURCE: National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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Achievement-Level Results 
by Jurisdiction 

Like the national results, achievement-level 
results for jurisdictions are presented in two 
ways: the percentage of students within 
each science achievement-level range, and 
the percentage of students at or above the 
Proficient level. The percentage of students 
within each science achievement-level 
range in 2000 by jurisdiction is presented 
in figure 2.10 for grade 4 and figure 2.11 
for grade 8. The shaded bars represent the 
proportion of students in each of the three 
achievement levels {Basic^ Proficient, and 
Advanced) as well as the proportion of 
students who are below Basic. Each popula- 
tion of students is aligned at the point 
where the Proficient level begins, so that 
scanning down the horizontal bars allows 
for easy comparison of the percentages of 
students who were at or above Proficient. 
Jurisdictions are listed in the figures in 
three clusters based on a statistical com- 
parison of the percentage of students at or 
above Proficient in each jurisdiction with 
the national percentage of public school 



students who were at or above Proficient. 
The cluster of jurisdictions at the top of 
each figure had a higher percentage of 
students at or above the Proficient level 
compared to the nation. For jurisdictions in 
the middle cluster, the percentages of 
students did not differ significantly from 
the national percentage. Jurisdictions in the 
bottom cluster had percentages lower than 
the national percentage. Within each 
cluster, jurisdictions are listed in alphabeti- 
cal order. 

Figure 2.10 shows that at grade 4, 12 
jurisdictions had higher percentages of 
students at or above Proficient than the 
nation, 17 had percentages that were not 
different from the nation, and 15 had 
percentages that were lower than the 
nation. In figure 2.11, the results for grade 
8 show 17 jurisdictions with higher per- 
centages of students at or above Proficient 
than the nation, 8 with percentages that 
were not different from the nation, and 17 
with percentages that were lower than the 
nation. 
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Figure 2.10 


Percentage of students within each science achievement-level range by state 


State Achievement- 


for grade 4 public schools: 2000 


Level Results, Grade 4 





The bars below indicate the percentages of students in each NAEP science achievement level. Each population of students is aligned 
at the point where the Proficient level begins, so that they may be compared at Proficient ar\6 above. States are listed alphabetically 
within three groups: the percentage at or above Proficient is higher than, not significantly different from, or lower than the nation. 
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^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

A Percentage is between 0.0 and 0.5. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 
NOTE: Numbers may not add to 100 due to rounding. National results are based on the national sample, not on aggregated state assessment samples. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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state Achievement- 
Level Results, Grade 8 



Percentage of students within each science achievement-level range hy state 
for grade 8 public schools: 2000 



The bars below indicate the percentages of students in each NAEP science achievement level. Each population of students is aligned 
at the point where the Proficient level begins, so that they may be compared at Proficient an6 above. States are listed alphabetically 
within three groups: the percentage at or above Proficient is higher than, not significantly different from, or lower than the nation. 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Tables 2.3 and 2.4 present the percentages 
of students performing at or above the 
Proficient level by jurisdiction for grades 4 
and 8, respectively. At grade 4, the percent- 
age of students at or above Prof dent ranged 
from less than 1 percent to 43 percent of 
students in 2000. At grade 8, the percent- 
age of students at or above the Proficient 



level ranged from 2 percent to 46 percent 
in 2000. Of the 36 jurisdictions that par- 
ticipated in both 1996 and 2000 at grade 8, 
6 made gains in the percentage of students 
at or above Proficient: Kentucky, Missouri, 
Vermont, WestVirginia, and the Depart- 
ment of Defense domestic schools and 
overseas schools (DDESS and DoDDS). 
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Table 2.3 State Proficient Lbvb\ Results, Grade 4 



Percentage of students at or above the Proficient level in science by state for grade 4 public schools: 

2000 



Nation 


28 


Alabama 


22 


Arizona 


22 


Arkansas 


24 


California ^ 


14 


Connecticut 


35 


Georgia 


23 


Hawaii 


16 


Idaho ^ 


30 


Illinois ^ 


31 


Indiana ^ 


32 


Iowa ^ 


37 


Kentucky 


29 


Louisiana 


19 


Maine + 


38 


Maryland 


26 


Massachusetts 


43 


Michigan + 


33 


Minnesota ^ 


35 


Mississippi 


14 


Missouri 


35 


Montana ^ 


37 


Nebraska 


26 


Nevada 


19 


New Mexico 


18 


New York + 


26 


North Carolina 


24 


North Dakota 


38 


Ohio + 


31 


Oklahoma 


26 


Oregon + 


28 


Rhode Island 


27 


South Carolina 


21 


Tennessee 


26 


Texas 


24 


Utah 


32 


Vermont ^ 


39 


Virginia 


33 


West Virginia 


25 


Wyoming 


33 


Other Jurisdictions 




American Samoa 


A 


DDESS 


29 


DoDDS 


30 


Guam 


4 


Virgin Islands 


4 



t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

A Percentage is between 0.0 and 0.5. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: National results are based on the national sample and not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 2.4 State Proficient Le^e\ Results, Grade 8 



Percentage of students at or above the Proficient level in science by state for grade 8 public schools: 
1996 and 2000 





1996 


2000 


Nation 


27 


30 


Alabama 


18 * 


22 


Arizona ^ 


23 


24 


Arkansas 


22 


23 


California ^ 


20 


15 


Connecticut 


36 


35 


Georgia 


21 


23 


Hawaii 


15 


15 


Idaho t 


— 


38 


Illinois ^ 


— 


30 


Indiana ^ 


30 


35 


Kentucky 


23 * 


29 


Louisiana 


13 * 


18 


Maine ^ 


41 


37 


Maryland 


25 


28 


Massachusetts 


37 * 


42 


Michigan ^ 


32 


37 


Minnesota ^ 


37 


42 


Mississippi 


12 


15 


Missouri 


28 t 


36 


Montana ^ 


41 


46 


Nebraska 


35 


36 


Nevada 


— 


23 


New Mexico 


19 


20 


New York ^ 


27 


30 


North Carolina 


24 


27 


North Dakota 


41 


40 


Ohio 


— 


41 


Oklahoma 


— 


26 


Oregon ^ 


32 


33 


Rhode Island 


26 


29 


South Carolina 


17 


20 


Tennessee 


22 


25 


Texas 


23 


23 


Utah 


32 


34 


Vermont ^ 


34 t 


40 


Virginia 


27 


31 


West Virginia 


21 t 


26 


Wyoming 


34 


36 


Other Jurisdictions 


American Samoa 


— 


2 


DDESS 


27 t 


35 


DoDDS 


31 t 


37 


Guam 


7 


6 



* Significantly different from 2000 if only one jurisdiction or the nation is being examined. 

^ Significantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on all jurisdictions that 
participated both years. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

— Indicates that the jurisdiction did not participate. 

NOTE: National results are based on the national sample and not on aggregated state assessment samples. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Cross-State Achievement Level 
Comparisons 

Figures 2.12 and 2.13 display the same type 
of state comparisons presented earlier for 
scale score results, but this time the perfor- 
mance measure being compared is the 
percentage of students at or above the 
Proficient achievement level for grades 4 and 
8, respectively. 



At grade 4, there were five states that 
had higher percentages of students at or 
above Proficient than the other states, but for 
which no significant differences were 
observed between them: Iowa, Maine, 
Massachusetts, Montana, and Vermont. At 
grade 8, the highest percentages of students 
at or above Proficient were in Massachusetts, 
Minnesota, Montana, and Ohio, which 
were not found to differ significantly from 
one another. 
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Figure 2.12: Cross-State Achievement Level Comparisons, Grade 4 



Comparisons of percentage of students at or above Proficient in science for grade 4 public schools: 2000 



Instructions: Read ioym the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction's abbreviation to the key belov\^ to determine whether the percentage of students at or above Proficient \n this jurisdiction is 
higher than, or lower than the jurisdiction in the column heading. For example, in the column under Michigan, the percentage of students in Michigan was 
lower than Massachusetts, all the states from Vermont through Oregon, and higher than the remaining states down the column. 
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Jurisdiction has statistically significantly higher percentage 
than the jurisdiction listed at the top of the figure. 

No statistically significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statistically significantly lower percentage 
than the jurisdiction listed at the top of the figure. 



BEST COPYAVAfUBLE 



The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Significance is determined by an 
application of a multiple comparison procedure (see appendix A). 

-»-»-lndicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE; Differences between states and jurisdictions may be partially 
explained by other factors not included in this figure. 

SOURCE: National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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HAPTER 2 



SCIENCE REPORT CARO 





Figure 2.13: Cross-State Achievement Level Comparisons, Grade 8 



Comparisons of percentage of students at or above Proficient in science for grade 8 public schools: 

2000 



Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction’s abbreviation to the key below to determine whether the percentage of students at or above Proficient in this jurisdiction 
is higher than, or lower than the jurisdiction in the column heading. For example, in the column under Michigan, the percentage of students in 
Michigan was lower than Montana, ail the states from Massachusetts through Illinois, and higher than the remaining states down the column. 
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Jurisdiction has statistically significantly higher percentage 
than the jurisdiction listed at the top of the figure. 

No statistically significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statistically significantly lower percentage 
than the jurisdiction listed at the top of the figure. 
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The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Significance is determined by an 
application of a multiple comparison procedure (see appendix A). 

-F+lndicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE; Differences between states and jurisdictions may be partially 
explained by other factors not included in this figure. 

SOURCE; National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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^ Subgroup Results for the 
Nation and the States 



This chapter presents the NAEP 2000 science results for 
various subgroups of students at both the national and state 
levels. National average scale score and achievement-level 
results are presented by six demographic characteristics: 

gender, race/ethnicity, parents’ education level, type 
of school, school location, and ehgibihty for the 
federal free/ reduced-price school lunch program. 
State results at grades 4 and 8 are presented for 
gender, race/ ethnicity, and eligibihty for the free/ 
reduced-price school lunch program. Additional 
information by subgroup for each jurisdiction that 
participated in the 2000 science assessment is 
available on the NAEP web site at http:/ / 
nces.ed.gov/ nationsreportcard. 

The differences that are reported in this chapter for 
demographic subgroups are based on statistical tests 
that consider both the magnitude of the difference 
between group average scores or percentages and 
the standard error of those statistics. Differences between 
groups and between assessment years are discussed only if 
they have been determined to be statistically significant. 
Within the sections summarizing achievement level results, 
only significant differences detected at or above Basic and 
Proficient are discussed in the text. Significant differences 
detected within achievement levels are not discussed, 
although they are shown in the figures. The reader should 
bear in mind that differences in science performance most 
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likely reflect a range of socioeconomic and 
educational factors that are not addressed in 
this report or by NAEP. 

National Results: 

Performance of 
Selected Subgroups 
Gender 

Gender differences in science achievement 
on large-scale school assessments have been 
examined at the international, national, and 
state level. The Third International Math- 
ematics and Science Study (TIMSS) that 
\vas conducted in 1995 reported that, at the 
fourth-grade level, males outperformed 
females in about one-half of the countries 
that participated including the U.S. At the 
eighth-grade level, vv^hile many of the 
countries that participated shovs^ed males 
outperforming females, this v^as not true 
for the U.S.; no diflference in performance 
M^as seen.^ At the twelfth-grade level, 
however, where mathematics and science 
literacy were tested, males outperformed 
females in most countries including the 
U.S.^ A repeat of TIMSS at the eighth- 
grade in 1999 (TIMSS-R) showed that 
males outperformed females in nearly half 
of the 38 countries, including the United 
States.^ 



In addition to international data about 
the performance of male and female 
students on science assessments, national 
studies such as NAEP also show male and 
female diflferences. For example, the 1990 
NAEP science assessment reported that 
males outperformed females at grade 8 and 
12; but found no diflference at grade 4^ In 
1996, when a new NAEP science assess- 
ment was administered, these results 
showed that males outperformed females at 
the twelfth-grade level only.^ The NAEP 
science assessment administered in 1996 
was also administered in 2000; thus a 
measure of performance by males and 
females on the same assessment can be 
obtained. 

Figure 3.1 presents the average science 
scores in 1996 and 2000 for male and 
female students at grades 4, 8, and 12. 

While average scores for males at grade 8 
were higher in 2000 than in 1996, average 
scores for twelfth-grade males were lower 
in 2000. None of the apparent changes 
across years in females’ average scores were 
statistically significant at any grade. In 2000, 
males had higher scores than females at 
grades 4 and 8, but the apparent difference 
between male and female students at grade 
12 was not statistically significant. 
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Mathematics and Science Study at the eighth grade. Chestnut Hill, MA: International Study Center, Lynch School of 
Education, Boston College. 

Gonzales, P., Calsyn, C., Jocelyn, L., Mak, K., Kastberg, D., Arafeh, S., Willi ams, T, &Tsen,W. (2000). Pursuing 
excellence: Comparisons of international eighth-grade mathematics and science achievement from a U.S. perspective, 1995 and 
1999 (NCES Publication No. 2001-028). Washington, DC: U.S. Department of Education, Office of Educational 
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DC: Office of Educational Research and Improvement. 
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Figure 3.1 



Average science scale scores by gender, grades 4, 8, and 1 2: 1 99B and 2000 



National Scale Score 
Results by Gender 




□ Male • Female 



★ Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Figure 3.2 provides a display of the 
science score gap between male and female 
students in 1996 and 2000. Even though 
the individual changes in average scores for 
male and female students at grade 4 were 
not statistically significant, taken together 
they created a significant difference favor- 
ing males over females. The increase in 
average scores among male students at 
grade 8 contributed to the creation of a 
similar difference favoring males at this 
grade level. Although the apparent narrow- 
ing of the gap between male and female 
twelfth-graders’ science scores was not 



statistically significant, their average scores 
in 2000 did not differ significantly as they 
did in 1996. 

These score gaps, and the score gaps 
presented in the following section between 
selected racial/ethnic subgroups, should be 
interpreted with caution. The average score 
of a selected subgroup does not represent 
the entire range of performance within 
that group. Furthermore, differences be- 
tween groups of students cannot be attrib- 
uted solely to group identification, as a 
variety of educational and social factors can 
affect student performance. 
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Figure 3.2 


Differences in average science scale scores by gender, grades 4, 8, and 12: 


National Scale Score 


1996 and 2000 


Differences by Gender 





Male Score Minus Female Score 





Grade 4 


Grades 


Grade 12 
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★ Significantly different from 2000. 

NOTE: Score differences are calculated based on differences between unrounded average scale scores. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Another way of looking at student 
performance is to examine the percentages 
of male and female students at or above 
each science achievement level. These 
results are presented in figure 3.3. At grade 
4, none of the apparent changes between 
1996 and 2000 in the percentages of male 
or female students at or above any of the 
achievement levels was statistically signifi- 
cant. At grade 8, the percentage of male 
students at or above Proficient increased 



from 31 percent in 1996 to 36 percent in 
2000. At grade 12, the percentage of male 
students at or above Basic decreased from 
60 percent in 1996 to 54 percent in 2000. 

Comparing the performance of males 
and females on the 2000 assessment shows 
a higher percentage of males than females 
at or above Proficient at aU three grade 
levels, and a higher percentage of males at 
or above Basic at grades 4 and 8. 
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Figure 3.3 


Percentages of students within each science achievement-level range and at or above 


National Achievement- 


achievement levels by gender, grades 4, 8, and 1 2: 1 996 and 2000 


Level Results by 




Gender 





Grade 4 



Advanced 

Proficient 

Basic 

Below 

Basic 





At or above 
Proficient 



At or above 
Basic 



Advanced 3% 



Female 





At or above 
Proficient 



At or above 
Basic 



Grade 8 



Advanced 4% 
Proficient 



Male 



27 %^ 



31 %^ 



Basic 



Below 

Basic 



31%^1 62% 



'96 
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★ Significantly different from 2000. 

NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Race/Ethnicity 

NAEP assessments in all subject areas 
consistently report student achievement by 
race/ethnicity as well as by differences in 
performances among various racial/ethnic 
groups. 

The differences provide important 
information about the progress being made 
to ensure that all students are making 
progress in a particular subject area. In 
order to collect data for this analysis, 
students who participated in the assessment 
were asked to indicate which of the fol- 
lowing racial/ ethnic subgroups best de- 
scribed them: White, Black, Hispanic, 
Asian/Pacific Islander, or American Indian 
(including Alaskan Native). Figure 3.4 
presents average scale scores for students by 
these subgroups at grades 4, 8, and 12. Data 
for Asian/Pacific Islander students were not 
reported for the 2000 science assessment at 
grade 4 because special analyses raised 
concerns about the accuracy and precision 
of these results.^ 

At grade 4, none of the apparent changes 
between 1996 and 2000 in the average 
scores of each racial/ ethnic subgroup were 



statistically significant. At grade 8, American 
Indian students’ average scores declined. At 
grade 12, White students had lower average 
scores in 2000 than in 1996. 

When students’ performance in 2000 
was compared across subgroups, differences 
in average scores were found at all three 
grade levels. At grade 4, White students 
scored higher, on average, than American 
Indian, Hispanic, or Black students. In 
addition, American Indian students scored 
higher on average than Hispanic or Black 
students. At grade 8, White students had 
higher average scores than any of the other 
subgroups. Eighth-grade Asian/Pacific 
Islanders scored higher, on average, than 
American Indian, Hispanic, or Black 
students. American Indian and Hispanic 
eighth-graders scored higher on average 
than Black eighth-graders. At grade 12, 
White and Asian/Pacific Islander students 
both had higher average scores than 
American Indian, Hispanic, or Black 
students. American Indian students had a 
higher average score than that of either 
Hispanic or Black students. 



^ See appendix A. 
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Average science scale scores by race/ethnicity, grades 4, 8, and 12: 1996 and 2000 



Figure 3.4 

National Scale Score 
Results by Race/ 
Ethnicity 




• White ■ Hispanic 

□ Asian/Pacific isiander o Biack 
A American indian 



★ Significantly different from 2000. 

♦ Special analyses raised concerns about the accuracy and precision of national grade 4 Asian/Pacific Islander results in 2000. As a result, they are omitted from 
the body of this report. See appendix A for a more detailed discussion. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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The average score gaps between White 
and Black students and between White and 
Hispanic students are shown in figure 3.5. 
Unlike the small gaps seen between male 
and female students, the size of the score 



gaps between these racial/ ethnic subgroups 
are much larger. None of the apparent 
difierences in these gaps between 1996 
and 2000 were found to be statistically 
significant. 



Figure 3.5 


Differences in average science scale scores by race/ethnicity, grades 4, 8, and 12: 


National Scale Score 


1996 and 2000 


Differences by Race/ 




Ethnicity 





White Score Minus Biack Score 
Grade 4 Grades Grade 12 




White Score Minus Hispanic Score 
Grade 4 Grades Grade 12 




NOTE: Score differences are calculated based on differences between unrounded average scale scores. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Achievement-level results for the racial/ 
ethnic groups are presented in figure 3.6. 
Although White twelfth-graders did show a 
decline in the percentage of students at or 
above between 1996 and 2000^ none 
of the apparent changes in the percentages 
of other racial/ ethnic subgroups at or 
above the Basic or Proficient levels were 



found to be statistically significant. When 
the performance of students in different 
racial/ethnic subgroups was compared in 
2000, a higher percentage ofWhite and 
Asian/Pacific Islander students were found 
to be at or above Basic and Proficient, com- 
pared to the other subgroups. This finding 
was consistent across the three grades. 



Figure 3.6a 


Percentages of students within each science achievement-level range and at or above 


National Achievement- 


achievement levels by race/ethnicity, grade 4: 1996 and 2000 


Level Results by 




Race/Ethnicity 
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See footnotes at end of figure. ► 
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Figure 3.6b 



Percentages of students within each science achievement-level range and at or above 
National Achievement- achievement levels by race/ethnicity, grade 8: 1 99B and 2000 
Level Results by Race/ 

Ethnicity (continued) 
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See footnotes at end of figure. ► 
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Figure 3.6c 

National Achievement- 
Level Results by Race/ 
Ethnicity (continued) 



Percentages of students within each science achievement-level range and at or above 
achievement levels by race/ethnicity, grade 1 2: 1 996 and 2000 



White 



Black 



Advanced 3% 
Proficient 






'96 



'00 



At or above 
Proficient 

At or above 
Basic 



Advanced 
Proficient (6%) 
Basic 



Below 

Basic 



1 % 



Hispanic 



n 


7 % 1 
33 % 







(6%)C 



'96 



7 % I 



At or above 
Proficient 




Below 

Basic 



Asian/Pacific Islander 



Advanced 3% 
Proficient I 
Basic 



22 % 



34% 56 % 



'96 



4% 



^2% 


26 % 






59 % 







At or above 
Proficient 



At or above 
Basic 



'00 



American Indian 



Advanced A 



1 % 




At or above 
Proficient 



At or above 
Basic 



★ Significantly different from 2000. 

A Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 

♦ Special analyses raised concerns about the accuracy and precision of national grade 4 Asian/Pacific Islander results in 2000. As a result, they are omitted from 
the body of this report. See appendix A fora more detailed discussion. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Parents’ Highest Level of Education 

It has been documented that, in general, 
higher levels of parental education are 
associated with higher levels of student 
performance.^ This has been noted not 
only in the U.S., but also in a number of 
other countries around the world. ^ 

Students who participated in the NAEP 
science assessment were asked to indicate 
the highest level of education completed 
by each parent. Four levels of education 
were identified: did not finish high school, 
graduated from high school, some educa- 
tion after high school, and graduated from 
college. Students could also choose the 
response, “I don’t know.” For this analysis, 
the highest education level reported for 
either parent was used. Data are presented 
for students in grades 8 and 12 only. Data 
were not collected at grade 4 because in 
previous NAEP assessments fourth-graders’ 
responses about their parents’ education 
were highly variable and contained a large 
percentage of“I don’t know” responses. 

The average science score results for aU 
levels of student-reported parent education 
are presented in figure 3.7. Almost one-half 
of both eighth-graders and twelfth-graders 



(47 and 48 percent, respectively) reported 
that at least one parent had graduated from 
college, whereas only 6 percent of both 
eighth- and twelfth-graders reported that 
their parents had not graduated from high 
school. Additional information on the 
percentages of students reporting different 
levels of parents’ education is available in 
appendix B. 

Comparisons of average scores by 
parental education across years show a 
dechne between 1996 and 2000 among 
twelfth-grade students whose parents’ 
highest level of education was high school 
graduation or some education after high 
school. Comparing students’ performance 
by level of parents’ education in 2000 
showed that eighth- and twelfth-graders 
whose parents graduated from college had 
higher scores, on average, than their peers 
whose parents had lower levels of educa- 
tion. In general, students who reported 
higher levels of parental education had 
higher average scores than their peers who 
reported lower levels of parental education. 
These results are consistent with the results 
from other studies. 



^ Braswell, J.S.,Lutkus,A.D., Grigg,W.S., Santapau, S.L.,Tay-Lim,B. S.-H., & Johnson, M.S. (2001). The nation^s report 
card: Mathematics 2000 (NCES Publication No. 2001-5 17). Washington, DC: U.S. Department of Education, Office 
of Educational Research and Improvement, National Center for Education Statistics. 

Donahue, P. L.,Voelkl, K.E., Campbell,J.R., & Maz 2 eo,J. (1999). NAEP 1998 reading report card for the nation and 
the states (NCES Publication No. 1999-500). Washington, DC: U.S. Department of Education, Office of Educa- 
tional Research and Improvement, National Center for Education Statistics. 

® Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., Gregory, K.D, Smith,T.A., Chrostowski, S.J., Garden, R.A., & 
O’Connor, K.M. (2000). TIMSS 1999 international science report: Findings from lEA's repeat of theThird International 
Mathematics and Science Study at the eighth grade. Chestnut Hill, MA: International Study Center, Lynch School of 
Education, Boston College. 
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Average science scale scores by student^reported parents' highest level of education, 
grades 8 and 12: 1996 and 2000 



Figure 3.7 

National Scale Score 
Results by Parents’ 
Education 




• Graduated college ■ Less than high school 

□ Some education O Unknown 

after high school 

A Graduated high school 

★ Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Achievement-level results across years by 
level of parental education are presented in 
figure 3.8 and show patterns similar to 
those found for average scale scores. None 
of the apparent changes between 1996 and 
2000 in percentages of eighth-grade 
students attaining achievement levels were 
statistically significant at any level of paren- 
tal education. Among twelfth-graders, 
however, a drop in performance is evident 
at the two highest levels of parental educa- 
tion. The percentage of twelfth-graders at 



or above Basic decreased between 1996 and 
2000 among those students whose parents’ 
highest level of education was some educa- 
tion after high school and among those 
students with at least one parent who 
graduated from college. 

Comparing students’ performance by 
parents’ level of education in 2000 shows a 
consistent pattern at grades 8 and 12. At 
both grades, the level of parents’ education 
had a positive relationship to the percentage 
of students at or above Basic and Proficient. 



Figure 3.8a 

National Achievement- 
Level Results by 
Parents’ Education 



Percentage of students within each science achievement-level range and at or 
above achievement levels by parents' highest level of education, grade 8: 

1996 and 2000 



Less than high school 



A(tvanced A A 




Graduated high school 



At or above 


Advanced 


Proficient 
At or above 


Proficient 


Basic 


Basic 




Below 




Basic 




18 % 



49 % 



1 % 

17 % 



18 % 



29 % I 46 % 



At or above 
Proficient 

At or above 
Basic 



Some education after high school 



Advanced 3% 3% 




At or above 
Proficient 

At or above 
Basic 



Graduated college 



Advanced 5 % 



7 % 




At or above 
Proficient 

At or above 
Basic 



Unknown 

Advanced A 1 % 
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Figure 3.8b 


Percentage of students within each science achievenienWevel range and at or 


National Achievement- 


above achievement levels by parents’ highest level of education, grade 12: 


Level Results by 
Parents' Education 


1998 and 2000 


(continued) 





Less than high school 



Advanced 
Proficient (3%) 

Basic 


A 






25 % 


Below 

Basic 








fW] 

'96 






At or above 
Proficient 

At or above 
Basic 



Graduated high school 




At or above 
Proficient 

At or above 
Basic 



Some education after high school 



Advanced 2% 
Proficient 





Graduated college 



Advanced 4% 



Proficient 



4% 




23% 



27% 



38% I 65% 




'00 



At or above 
Proficient 



At or above 
Basic 



Unknown 



Advanced 



'96 



Proficient (4%) 
Basic 




I I (3%) 

17% \ 




18% 1 


Below 

Basic 












I 83'i 




[«%] 





'00 



At or above 
Proficient 
At or above 
Basic 



★ Significantly different from 2000. 

A Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Type of School 

The schools that participate in the NAEP 
assessment are classified as either public or 
nonpublic. ^ Differences in performance on 
NAEP science assessments between stu- 
dents attending public and nonpublic 
schools typically show students attending 
nonpublic schools outperforming their 
public school peers, on average.^® It is 
worth noting, however, that results from a 
special study of twelfth-grade students 
taking advanced science courses showed 
that the performance of twelfth-graders in 
public schools who were enrolled in an 
advanced science course was not found to 
be significantly different from that of 
twelfth-graders taking advanced science 
courses in nonpublic schools. Despite the 
general pattern of nonpublic school stu- 
dents outperforming public school stu- 
dents, readers are cautioned to consider the 
possibility that socioeconomic and socio- 
logical factors related to type of school 
enrollment may affect student performance. 
These factors are not accounted for in the 
NAEP assessment results. 



Nine out of ten students who partici- 
pated in the 2000 NAEP science assess- 
ment attended public schools (89 percent 
at grade 4, 90 percent at grade 8, and 91 
percent at grade 12). Additional informa- 
tion on the percentages of students attend- 
ing public and nonpublic schools can be 
found in appendix B. Figure 3.9 presents 
the average science scores by type of 
school. None of the apparent changes 
between 1996 and 2000 in average scores 
of fourth- and eighth-graders attending 
either public or nonpublic schools were 
statistically significant. At grade 12, how- 
ever, average scores for students attending 
nonpublic schools increased from 155 in 
1996 to 161 in 2000, while scores for 
students attending public schools decreased 
from 149 to 145. 

A comparison of students’ average score 
by type of school attended in 2000 contin- 
ues the trend found in other NAEP assess- 
ments; fourth- eighth-, and twelfth-graders 
who attended nonpublic schools had 
higher scores, on average, than their peers 
who attended public schools. 



’ More details on results by school type including additional breakouts by types of nonpublic schools are available at 
the NAEP web site (http://nces.ed.gov/nationsreportcard). 

Campbell, J.R.,Voelkl, K.E., & Donahue, P.L. (1997). NAEP 1996 trends in academic progress (NCES Publication 
No. 97— 985). Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement, 
National Center for Education Statistics. 

Campbell, J.R., Hombo, C.M., & Mazzeo, J. (2000). NAEP 1999 trends in academic progress: Three decades of student 
performance (NCES Publication No. 2000-469). Washington, DC: U.S. Department of Education, Office of 
Educational Research and Improvement, National Center for Education Statistics. 

” O’Sullivan, C.Y., & Grigg,WS. (2001). Assessing the best: NAEP^s 1996 assessment of twelfth-graders taking advanced 
science courses (NCES Publication No. 2001— 451). Washington, DC: U.S. Department of Education, Office of 
Educational Research and Improvement, National Center for Education Statistics. 
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Figure 3.9 



Average science scale scores by type of school, grades 4, 8, and 1 2: 1 996 and 2000 



National Scale Score 
Results by Type of 
School 



Grade 4 ■ Grade 8 ■ Grade 12 

'96 '00 I '96 '00 H '96 '00 




• Nonpublic □ Public 



★ Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Achievement-level results by school type 
are presented in figure 3.10. At grades 4 
and 8, none of the apparent changes be- 
tween 1996 and 2000 in percentages of 
either public or nonpublic school students 
at or above Basic, at or above Proficient, or at 
Advanced were statistically significant. At 
grade 12, however, the results for public 
school and nonpublic school students show 
opposite trends in attainment of the Basic 
and Projicient achievement levels. Between 
1996 and 2000, the percentage of public 
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school twelfth-graders at or above Basic and 
at or above Projicient decreased, while the 
percentages of their nonpublic school peers 
attaining these achievement levels in- 
creased. 

Comparing students’ performance by 
type of school in 2000 shows a consistent 
pattern at grades 4, 8, and 12. At all three 
grades, a greater percentage of nonpublic 
school students than public school students 
were at or above Basic, at or above Projicient, 
or at Advanced. 
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Figure 3.10 



Percentage of students within each science achievement-level range and at 
National Achievement achievement levels by type of school, grades 4, 8, and 1 2: 1 996 and 2000 

Level Results by Type 
of School 



Grade 4 



Adv3nced 3% 



Public 



Proficient 

Basic 

Below 

Basic 




3% 



At or above 
Proficient 



At or above 
Basic 










24% 


28 % 




'00 



Advanced 5% 



Nonpublic 

5% 




At or above 
Proficient 



At or above 
Basic 



Grade 8 



Public 

Advanced 3% 




'96 



4% 



At or above 
Proficient 



At or above 
Basic 






'00 




Nonpublic 



Advanced 5% 



7% 




At or above 
Proficient 

At or above 
Basic 



Grade 12 

Advanced 3% 
Proficient | 18% 
Basic 



Public 





Nonpublic 

Advanced 2% 

Proficient 



Basic 



Below 
Basic 

'96 





At or above 
Proficient 



At or above 
Basic 



★ Significantly different from 2000. 

NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Type of Location 

The schools from which NAEP draws its 
samples of students are classified according 
to their type of location. Based on Census 
Bureau definitions of metropohtan statisti- 
cal areas, including population size and 
density, the three mutually exclusive cat- 
egories are: central city, rural/small town, 
and urban fringe/large town. Because of 
slight changes by the Census Bureau in the 
definitions of these categories, schools were 
not classified in exactly the same way in 
2000 as in previous years in terms of 
location type. Therefore, comparisons to 
previous years are not possible, and only 



the data for the 2000 assessment are re- 
ported. More information on the defini- 
tions of the 2000 assessment classifications 
of location type is given in appendix A. 

Average science scale scores for fourth-, 
eighth-, and twelfth-grade students attend- 
ing schools in the three different types of 
location are presented in table 3.1. At 
grades 4 and 8, students in central city 
locations had lower average scores than 
students in urban fringe/large town or 
rural/small town locations. At grade 12, 
there was no statistically significant rela- 
tionship between school location and 
student performance. 



Table 3.1 National Scale Score Results by Type of Location 



Average science scale scores by type of location, grades 4, 8, and 12: 2000 



Grade 4 



Central city Urban fringe/large town Rural/small town 



140 



155 



152 



Graded 


142 


156 


152 




Grade 12 


144 


149 


145 





SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Percentages of students within and at or 
above each achievement level by type of 
school location are presented in figure 
3.11. At grades 4 and 8, the percentages of 
students at or above Basic and Proficient 
were higher in urban ffinge/large town 
and rural/ small town locations than central 
city locations. The percentage of fourth- 



graders at Advanced was also higher among 
students in urban fringe locations than in 
central cities. At grade 12, there were no 
statistically significant differences in the 
percentages of students at or above Basic or 
Proficient or at Advanced based on the 
school’s location. 



Figure 3.11 

National Achievement- 
Level Results by Type 
of Location 



Percentage of students within each science achievement-level range and at 
or above achievement levels by type of location, grades 4, 8, and 12: 2000 



Advanced 3% 
Proficient 



Grade 4 




Central city 



Urban fringe/ 
large town 



Rural/ 
small town 



Advanced 3 % 
Proficient 



1 % 



24% 



Basic 



Below 

Basic 



26% I 51% 



Grade 8 
5% 



32% 



36% 



[31% 67% 



4% 



29% 

5 



33% 



63% 



At or above 
Proficient 



At or above 
Basic 



Central city 



Urban fringe/ 
large town 



Rural/ 
small town 



Advanced 2% 



Proficient \ 15% 
Basic 



17% 



32% I 50% 



Below 

Basic 



Central city 



Grade 12 
2 % 




Urban fringe/ 
large town 



Rural/ 
small town 



NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Free/Reduced-Price School 
Lunch Eligibility 

Funded by the U.S. Department of Agri- 
culture (USDA) as part of the National 
School Lunch program, the free/reduced- 
price school lunch program is designed to 
assure that children at, near, or below the 
poverty line receive nourishing meals. 
Eligibility guidelines for the lunch program 
are based on the Federal income poverty 
guidelines and are stated by household 
size.^^ NAEP began collecting data on 
student eligibility for this program in 1996. 

As shown in figure 3.12, average science 
scores for students who were not eligible 
for the free/reduced-price school lunch 
program (i.e., those above the poverty 
guidelines) were higher than the scores for 
students who were eligible for the program. 
Since information on eligibility is not 



available for a substantial percentage of the 
students at each grade, scale score averages 
for this group of students are also provided. 
It should be noted that students for whom 
the information was not available (which 
included students from schools that did not 
offer free/ reduced-price school lunches) 
also had higher average scores at each of 
the three grades than the students who 
were eligible for the free/reduced-price 
school lunch program. 

Comparisons across years show lower 
average scores in 2000 than in 1996 among 
eighth-graders who were eligible for the 
program and higher scores among students 
who were not eligible. At grade 12, stu- 
dents who were not eligible for the pro- 
gram had lower average scores in 2000 
than in 1996. 



U.S. General Services Administration. (1999), Catalogue of federal domestic assistance. Washington, DC: Executive 
Office of the President, Office of Management and Budget. 
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Average science scale scores by student eligibility for free/reduced-price 
school lunch program, grades 4, 8, and 12: 1996 and 2000 



Figure 3,12 

National Scale Score 
Results by Free/Reduced* 
Price School Lunch 
Eligibility 




• Not Eligible □ Eligible A Info not available 

★ Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Achievement-level results by students’ 
eligibility for the free/reduced-price school 
lunch program are displayed in figure 3.13. 
At grade 4, there were no statistically 
significant changes between 1996 and 2000 
in the percentages of students at or above 
achievement levels among students who 
were either eligible or not eHgible for the 
free/ reduced-price school lunch program. 
At grade 8, the percentage of students at or 
above Proficient increased between 1996 
and 2000 for those students who were not 



eHgible for the free/reduced-price school 
lunch program. At grade 12, the percent- 
ages of students at or above Basic decreased 
between 1 996 and 2000 for those students 
who were not eligible for the fi:ee/reduced- 
price school lunch program. Similar to the 
pattern observed for scale score results in 
2000, there were higher percentages of 
fourth-, eighth-, and twelfth-graders at or 
above Basic and Proficient among those 
students who were not eHgible for the 
program than among those who were. 
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Figure 3.13 

National Achievement- 
Level Results by Free/ 
Reduced-Price School 
Lunch Program 
Eligibilty 



Percentage of students within each science achievemenWevel range and at 
or above achievement levels by student eligibility for the free/reduced-price 
school lunch program, grades 4, 8, and 12: 1996 and 2000 
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See footnotes at end of figure. ► 
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Figure 3.13 

National Achievement^ 
Level Results by Free/ 
Reduced-Price School 
Lunch Program 
Eligibilty (continued) 



Percentage of students within each science achievement-level range and at 
or above achievement levels by student eligibility for the free/reduced-price 
school lunch program, grades 4, 8, and 12: 1996 and 2000 



Grade 12 



Eligible 





At or above 
Proficient 

At or above 
Basic 



Advanced 

Proficient 

Basic 

Below 

Basic 



Not eligible 
3% 




'96 



2 % 




'00 



At or above 
Proficient 



At or above 
Basic 



Advanced 

Proficient 

Basic 

Below 

Basic 



Info not available 




2 % 




At or above 
Proficient 

At or above 
Basic 



★ Significantly different from 2000. 

▲ Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



State Results: Performance of 
Selected Subgroups 

Results for public schools in participating 
states and jurisdictions are presented in this 
section by gender, race/ethnicity, and 
ehgibihty for ffee/reduced-price school 
lunch. Complete data for participating 
jurisdictions are available on the NAEP 
web site at http://nces.ed.gov/ 
nationsreportcard. 

Nonpublic schools were not included in 
the state NAEP assessments for 2000, but 
were included in the national samples. 
While the national results shown in the 
previous sections of this chapter repre- 
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sented both public and nonpublic school 
students combined, the national data shown 
for comparison at the top of the following 
state tables are based on the national 
sample — not on aggregated state samples — 
of students from public schools only. 

In addition to results from the 2000 state 
assessment, results are also available from 
1996 for many of the jurisdictions at grade 
8. Not aU jurisdictions, however, met 
minimum school participation guidehnes 
in every NAEP assessment. (See appendix 
A for details on the participation and 
reporting guidehnes.) In 2000, results for 
grades 4 and 8 in Wisconsin and grade 8 in 
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the Virgin Islands are not included in the 
relevant tables because they did not meet 
the criteria. 

The state results presented here were 
obtained by assessing a representative 
sample of students in each state under 
conditions that did not permit accommo- 
dations for special-needs students. These 
were the same conditions under which 
results were obtained in previous state 
assessments. Consequently, it is possible to 
report changes in student performance 
across the assessment years at grade 8. In 
2000, a separate representative sample was 
assessed in each participating jurisdiction 
for which accommodations were offered to 
special-needs students. Those results are 
presented in chapter 4, along with a com- 
parison of“accommodations-permitted” 
and“accommodations-not-permitted” 
results in each state. Subgroup “accommo- 
dations-permitted” results by state are 
available on the NAEP web site. 

In examining the state results presented 
in this section, it should be noted that 
schools participating in the NAEP assess- 
ments under these conditions are permitted 
to exclude those students who cannot be 
assessed meaningfully without accommo- 
dations. Exclusion rates vary considerably 
across years in many jurisdictions. In 2000, 
in the sample that did not permit accom- 
modations, the pattern in most jurisdictions 
was for more special-needs students to be 
excluded from the assessment than in 1996. 

In addition to changes across years in 
exclusion rates for a particular jurisdiction, 
there is considerable variation in exclusion 
rates across jurisdictions. Comparisons of 
assessment results across jurisdictions and 
within jurisdictions across years should be 



made with caution. No adjustments have 
been made for differing exclusion rates 
across jurisdictions or across years. Thus, a 
comparison within a jurisdiction across 
years or between two jurisdictions may be 
based on samples with exclusion rates that 
differ considerably. The exclusion rates for 
each jurisdiction are presented in appendix A. 
Tables presenting state-level results at grade 
8 indicate statistically significant changes 
across years when examining only one 
jurisdiction at a time ('^), and when using a 
multiple comparison procedure based on 
all the jurisdictions that participated (t). 
Only those differences based on the mul- 
tiple comparison procedure are discussed. 

Gender Results by State 

Table 3.2 presents the results for the grade 
4 male and female average science scores 
for each jurisdiction that participated in the 
2000 assessment. Since this was the first 
time the assessment was given at the state 
level, there are no comparisons to other 
years. At grade 4, the average score of male 
students was higher than that of female 
students in 7 states and other jurisdic- 
tions — Connecticut, Georgia, Maine, 

North Dakota, Utah, Wyoming, and the 
Department of Defense Dependents 
Schools (Overseas) (DoDDS). 

Table 3.3 presents the results for the 
grade 8 male and female average science 
scores for each jurisdiction that participated 
in the 2000 assessment. For both males and 
females the 2000 average score is compared 
to scores from 1996, where available. The 
following discussion of changes in sub- 
group performance within jurisdictions is 
based only on results of the statistical 
testing using a multiple-comparison proce- 
dure. At grade 8, in 2000, the average score 
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Table 3.2 State Scale Score Results by Gender, Grade 4 



State average science scale scores by gender for grade 4 public schools: 2000 





Male 


Female 


Nation 


151 


146 


Alabama 


143 


143 


Arizona 


142 


140 


Arkansas 


145 


143 


California ^ 


132 


130 


Connecticut 


160 


153 


Georgia 


147 


140 


Hawaii 


138 


135 


Idaho ^ 


155 


150 


Illinois ^ 


154 


148 


Indiana ^ 


157 


153 


Iowa ^ 


163 


158 


Kentucky 


155 


150 


Louisiana 


141 


136 


Maine t 


165 


158 


Maryland 


148 


144 


Massachusetts 


164 


159 


Michigan ^ 


156 


151 


Minnesota ^ 


159 


155 


Mississippi 


135 


132 


Missouri 


159 


153 


Montana ^ 


163 


157 


Nebraska 


153 


148 


Nevada 


142 


142 


New Mexico 


140 


136 


New York ^ 


151 


147 


North Carolina 


150 


146 


North Dakota 


164 


156 


Ohio + 


156 


152 


Oklahoma 


153 


150 


Oregon ^ 


151 


148 


Rhode Island 


151 


145 


South Carolina 


143 


139 


Tennessee 


150 


145 


Texas 


150 


145 


Utah 


157 


152 


Vermont ^ 


161 


157 


Virginia 


157 


155 


West Virginia 


152 


149 


Wyoming 


162 


153 


Other Jurisdictions 


American Samoa 


52 


49 


DDESS 


158 


155 


DoDDS 


159 


153 


Guam 


108 


113 


Virgin Islands 


118 


113 



t Indicates that the jurisdiction did not meet one or more of the guideiines for schooi participation. 

NOTE; Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 3.3 State Scale Score Results by Gender, Grade 8 



State average science scale scores by gender for grade 8 public schools: 1996 and 2000 




1996 






2000 




Male 


Female 


Male 


Female 


Nation 


149* 


148 


153 


146 


Alabama 


138 


139 


144 


139 


Arizona ^ 


147 


143 


150 


142 


Arkansas 


147 


142 


144 


142 


California ^ 


140 


136* 


136 


129 


Connecticut 


156 


155 


158 


150 


Georgia 


144 


139 


147 


140 


Hawaii 


135 


135 


133 


131 


Idaho ^ 


— 


— 


162 


155 


Illinois ^ 


— 


— 


153 


148 


Indiana ^ 


154 


152 


158 


154 


Kentucky 


148* 


147 


155 


148 


Louisiana 


136 


129 


138 


134 


Maine ^ 


165 


161 * 


163 


157 


Maryland 


146* 


145 


152 


147 


Massachusetts 


159 


154* 


162 


160 


Michigan ^ 


156 


150 


158 


154 


Minnesota ^ 


161 


157 


162 


158 


Mississippi 


134 


132 


136 


132 


Missouri 


152* 


150* 


159 


154 


Montana ^ 


164 


160 


169 


161 


Nebraska 


160 


155 


160 


154 


Nevada 


— 


— 


145 


142 


New Mexico 


143 


139 


144 


137 


New York ^ 


148 


143 


151 


147 


North Carolina 


149 


145 


151 


144 


North Dakota 


163 


161 


163 


159 


Ohio 


— 


— 


164 


157 


Oklahoma 


— 


— 


152 


146 


Oregon ^ 


157 


153 


155 


153 


Rhode Island 


150 


148 


152 


147 


South Carolina 


141 


136 


145 


139 


Tennessee 


144 


142 


149 


143 


Texas 


147 


143 


147 


141 


Utah 


159 


154 


158 


153 


Vermont ^ 


158* 


156 


163 


159 


Virginia 


150* 


148 


156 


148 


West Virginia 


148* 


147 


153 


147 


Wyoming 


159 


156 


159 


156 


Other Jurisdictions 










American Samoa 


— 


— 


70 


75 


DDESS 


157 


149* 


160 


157 


DoDDS 


157* 


154 


162 


156 


Guam 


120 


120 


116 


112 



* Significantly different from 2000 if only one jurisdiction or the Nation is being examined. 

t Significantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on all jurisdictions that 
participated both years. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
OOESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS; Department of Defense Dependents Schools (Overseas). 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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of male students was higher than that of 
female students in 23 jurisdictions. Be- 
tween 1996 and 2000 gains were evident 
for males in three jurisdictions — Kentucky, 
Missouri, and the Department of Defense 
Dependents School (Overseas) (DoDDS). 
Gains were made by females in only one 
jurisdiction between 1996 and 2000 — the 
Department of Defense Domestic Depen- 
dent Elementary and Secondary Schools 
(DDESS). 

Tables 3.4 and 3.5 present the percent- 
age of males and females at or above 
Proficient for the participating jurisdictions 
at grades 4 and 8 respectively. At grade 4, 
the percentage of students at or above 
Proficient in 2000 was higher for male 
students than for female students in 



7 jurisdictions — Connecticut, Maine, 
North Dakota, Rhode Island, Utah, Wyo- 
ming, and the Department of Defense 
Dependents School (Overseas). At grade 8, 
the percentage of students at or above 
Proficient in 2000 was higher for male 
students than for female students in 29 
jurisdictions. When results from 1996 were 
compared to those of 2000, the percentage 
of male students at or above Proficient was 
higher in 2000 in 4 jurisdictions — Ken- 
tucky, Missouri, West Virginia, and the 
Department of Defense Dependents 
School (Overseas) (DoDDS) . There were 
no statistically significant changes between 
1996 and 2000 among female eighth- 
graders in any of the jurisdictions. 
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Table 3.4 State Proficient Level Achievement Results by Gender, Grade 4 



State percentages of students at or above the Proficient level in science by gender for grade 4 public 
schools: 2000 





Male 


Female 


Nation 


31 


24 


Alabama 


23 


21 


Arizona 


24 


20 


Arkansas 


26 


21 


California ^ 


16 


12 


Connecticut 


40 


30 


Georgia 


27 


20 


Hawaii 


18 


14 


Idaho ^ 


35 


25 


Illinois ^ 


34 


28 


Indiana ^ 


37 


28 


Iowa ^ 


42 


33 


Kentucky 


32 


25 


Louisiana 


22 


16 


Maine ^ 


43 


34 


Maryland 


29 


23 


Massachusetts 


46 


38 


Michigan ^ 


37 


29 


Minnesota ^ 


38 


32 


Mississippi 


16 


12 


Missouri 


39 


31 


Montana ^ 


43 


32 


Nebraska 


29 


23 


Nevada 


21 


17 


New Mexico 


20 


16 


New York ^ 


28 


24 


North Carolina 


26 


22 


North Dakota 


44 


32 


Ohio^ 


34 


29 


Oklahoma 


29 


24 


Oregon ^ 


29 


26 


Rhode Island 


31 


23 


South Carolina 


24 


17 


Tennessee 


29 


23 


Texas 


28 


21 


Utah 


36 


27 


Vermont ^ 


41 


36 


Virginia 


35 


30 


West Virginia 


26 


23 


Wyoming 


39 


27 


Other Jurisdictions 


American Samoa 


A 


A 


DDESS 


33 


26 


DoDDS 


35 


26 


Guam 


4 


4 


Virgin Islands 


4 


3 



A Percentage is between 0.0 and 0.5. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elemental^ and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 3.5 State Proficient \.eve\ Achievement Results by Gender, Grade 8 



State percentages of students at 


or above the Proficient level 


in science by gender for grade 8 


public schools: 1996 and 2000 




1996 






2000 






Male 




Female 


Male 


Female 


Nation 


29* 




26 


35 




26 


Alabama 


19 




17 


24 




20 


Arizona ^ 


25 




20 


29 




19 


Arkansas 


26 




18 


25 




21 


California ^ 


21 




18 


18 




13 


Connecticut 


37 




35 


39 




30 


Georgia 


24 




17 


27 




20 


Hawaii 


16 




14 


17 




14 


Idaho ^ 


— 




— 


44 




32 


Illinois ^ 


— 




— 


34 




26 


Indiana ^ 


32 




28 


38 




32 


Kentucky 


25* 




21 


34 




24 


Louisiana 


17 




10* 


21 




15 


Maine ^ 


45 




38 


42 




32 


Maiyland 


26 




24 


32 




25 


Massachusetts 


40 




33 


44 




40 


Michigan ^ 


36 




29 


38 




35 


Minnesota ^ 


40 




33 


45 




38 


Mississippi 


14 




11 


17 




12 


Missouri 


31* 




25* 


40 




32 


Montana ^ 


44 




37 


52 




39 


Nebraska 


39 




30 


41 




31 


Nevada 


— 




— 


25 




20 


New Mexico 


23 




16 


25 




16 


New York ^ 


31 




23 


32 




27 


North Carolina 


26 




22 


31 




23 


North Dakota 


44 




37 


44 




36 


Ohio 


— 




— 


46 




36 


Oklahoma 


— 




— 


31 




22 


Oregon ^ 


35 




29 


37 




30 


Rhode Island 


28 




24 


31 




26 


South Carolina 


20 




15 


23 




18 


Tennessee 


24 




20 


29 




21 


Texas 


27 




20 


27 




20 


Utah 


37 




27 


39 




30 


Vermont ^ 


36* 




32 


43 




36 


Virginia 


28 




26 


35 




27 


West Virginia 


22* 




19 


30 




22 


Wyoming 


35 




32 


39 




32 


Other Jurisdictions 














American Samoa 


— 




— 


3 




1 


DDESS 


32 




21 * 


38 




33 


DoDDS 


33* 




29 


42 




33 


Guam 


8 




7 


7 




5 


* significantly different from 2000 if only one jurisdiction or the Nation is being examined. 

t Sigriificantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on ail jurisdictions that 



participated both years. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Race/Ethnicity Results by State 

Tables 3.6 and 3.7 display the average 
science scores for each of the racial/ethnic 
groups by jurisdiction in 2000 for grade 4, 
and in 1996 and 2000 for grade 8. In every 
state and other jurisdiction where sample 
sizes were large enough for reliable statisti- 



cal comparisons, White students outper- 
formed Black and Hispanic students at 
both grades. There were no statistically 
significant differences detected between 
1996 and 2000 in any state or jurisdiction 
in the average scores of eighth-graders in 
the different racial/ethnic subgroups. 
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Table 3.6 State Scale Score Results by Race/Ethnicity, Grade 4 



State average science scale 


scores by 


race/ethnicity for grade 4 public schools: 2000 






White 


Black 


Hispanic 


Asian/ 

Pacific Islander 


American 

Indian 


Nation 


159 


124 


127 




139 


Alabama 


158 


125 


117 


kkkk 


kkkk 


Arizona 


157 


128 


123 


kkkk 


115 


Arkansas 


156 


117 


121 


kkkk 


144 


California ^ 


151 


119 


115 


142 


**** 


Connecticut 


166 


127 


133 


kkkk 


kkkk 


Georgia 


160 


124 


128 


162 


**** 


Hawaii 


148 


125 


119 


138 


**** 


Idaho t 


158 


**** 


126 


kkkk 


**** 


Illinois ^ 


166 


127 


129 


kkkk 


kkkk 


Indiana ^ 


160 


132 


130 


kkkk 


kkkk 


Iowa + 


162 


■k-k-k-k 


141 


kkkk 


**** 


Kentucky 


156 


129 


138 


kkkk 


kkkk 


Louisiana 


156 


121 


126 


kkkk 


**** 


Maine + 


163 


kkkk 


144 


kkkk 


kkkk 


Maryland 


162 


125 


133 


164 


134 


Massachusetts 


169 


137 


130 


161 


kkkk 


Michigan + 


164 


121 


132 


kkkk 


kkkk 


Minnesota + 


163 


126 


136 


134 


148 


Mississippi 


153 


117 


114 




**** 


Missouri 


164 


131 


129 


kkkk 


152 


Montana + 


164 


kkkk 


147 


kkkk 


145 


Nebraska 


155 


125 


136 


kkkk 


kkkk 


Nevada 


152 


121 


127 


147 


145 


New Mexico 


155 


129 


129 


**** 


123 


New York + 


163 


131 


132 


156 


kkkk 


North Carolina 


159 


128 


133 


**** 


132 


North Dakota 


163 


kkkk 


145 


kkkk 


136 


Ohiot 


161 


129 


141 


kkkk 


**** 


Oklahoma 


159 


133 


136 


kkkk 


148 


Oregon + 


156 


kkkk 


123 


kkkk 


148 


Rhode Island 


159 


121 


116 


143 


**** 


South Carolina 


157 


123 


128 


kkkk 


**** 


Tennessee 


157 


122 


128 


kkkk 


**** 


Texas 


162 


134 


135 


158 


kkkk 


Utah 


160 


kkkk 


135 


147 


138 


Vermont + 


160 


kkkk 


kkkk 


kkkk 


kkkk 


Virginia 


166 


139 


140 


176 


kkkk 


West Virginia 


152 


132 


135 


kkkk 


kkkk 


Wyoming 


161 


kkkk 


142 


kkkk 


149 


Other jurisdictions 


American Samoa 




kkkk 


36 


58 


kkkk 


DDESS 


166 


145 


154 


157 


kkkk 


DoDDS 


163 


141 


151 


156 


153 


Guam 


112 


kkkk 


88 


116 


**** 


Virgin Islands 


**** 


119 


106 


kkkk 


kkkk 



**** Sample size is insufficient to permit a reliable estimate. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 

~ Special analyses raised concerns about the accuracy and precision of the National grade 4 Asian/Pacific Islander results in 2000. As a result, they are omitted from the 
body of this report. See appendix A for a more detailed discussion. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 3.7 State Scale Score Results by Race/Ethnicity, Grade 8 



State average science scale scores by race/ethnicity for grade 8 public schools: 1996 and 2000 



White 



Black 



Hispanic 



Asian/Pacific Islander American Indian 





1996 


2000 


1996 


2000 


1996 


2000 


1996 


2000 


1996 


2000 


Nation 


159 


160 


120 


121 


127 


127 


150 


154 


148* 


132 


Alabama 


151 


154 


117 


116 


107 


106 


**** 


**** 


**** 


**** 


Arizona ^ 


157 


159 


124 


127 


129 


126 


**** 


**** 


121 


137 


Arkansas 


154 


154 


116 


113 


122 


118 


**** 


**** 


**** 


**** 


California ^ 


156 


150 


121 


120 


121 


117 


148 


147 


**** 


★★★★ 


Connecticut 


165 


166 


121 


122 


122 


129 


163 


160 


**** 


**** 


Georgia 


155 


159 


122 


123 


128 


124 


**** 


**** 


**** 


**** 


Hawaii 


146 


149 


128 


128 


119 


119 


136* 


132 


**** 


**** 


Idaho ^ 


— 


162 


— 


**** 


— 


135 


— 


**** 


— 


**** 


Illinois ^ 


— 


165 


— 


123 


— 


131 


— 


162 


— 


**** 


Indiana ^ 


158 


161 


125 


127 


139 


132 


**** 


**** 


**** 


**** 


Kentucky 


151 * 


155 


127 


126 


113 


**★* 


**** 


**** 




**★* 


Louisiana 


148* 


154 


113 


113 


104 


119 


**** 


**** 


**** 


**** 


Maine ^ 


164* 


161 


**** 


**** 


141 


**** 


**** 


**** 


**** 


**** 


Maryland 


160 


163 


124 


127 


121* 


135 


161 


170 


**** 


**** 


Massachusetts 


163* 


168 


126 


134 


126 


128 


152 


165 


**** 


**** 


Michigan ^ 


161 


164 


122 


120 


134 


137 


**** 




**** 


**** 


Minnesota ^ 


162 


165 


130 


122 


134 


136 


152 




**** 


**** 


Mississippi 


149 


150 


119* 


114 


105 


113 


★*** 




**** 


**** 


Missouri 


158* 


162 


120 


125 


130 


141 




**★* 


**** 


**** 


Montana ^ 


166 


168 


**** 


**** 


147 


151 






139 


143 


Nebraska 


161 


162 


130 


129 


134 


132 






**** 


**** 


Nevada 


— 


154 


— 


125 


— 


126 


— 


148 


— 


134 


New Mexico 


159 


160 


**** 


**** 


130 


130 




**** 


126 


124 


New York ^ 


161 


165 


120 


128 


116 


125 


155 


151 


**** 


**** 


North Carolina 


157 


158 


126 


123 


123* 


139 




158 


136 


**** 


North Dakota 


164 


164 


**** 


**** 


137 


139 






137 


133 


Ohio 


— 


165 


— 


131 


— 


147 


— 




— 


**** 


Oklahoma 


— 


156 


— 


127 


— 


123 


— 




— 


145 


Oregon ^ 


158 


160 


**** 


131 


133 


128 


157 


157 


142 


144 


Rhode Island 


155 


156 


130 


128 


118 


127 


142 


143 


**** 


**** 


South Carolina 


153 


155 


122 


122 


122 


123 






**** 


**** 


Tennessee 


151 


155 


117 


118 


104 


123 


**** 




**** 


**** 


Texas 


161 


159 


127 


122 


129 


132 


157 


162 




**** 


Utah 


159 


159 




**** 


133 


135 


143 


152 


**** 


**** 


Vermont 


159 


162 




**** 


136 


**** 


**** 


**** 




**** 


Virginia 


158 


161 


126 


130 


132 


138 


165 


169 




**** 


West Virginia 


149 


151 


127 


125 


122 




**** 


**** 


**** 


**** 


Wyoming 
Other Jurisdictions 


161 


161 






140 


139 




**** 


138 


141 


American Samoa 


— 


**** 






— 


55 


— 


90 


— 




DDESS 


162* 


169 


137 


140 


149 


156 




**** 




**** 


DoDDS 


164 


168 


140 


142 


146 


153 


156 


160 




*★** 


Guam 


138 








106 


97 


122 


119 







* Significantly different from 2000 if only one jurisdiction or the Nation is being examined. 

**** Sample size is insufficient to permit a reliable estimate. 

Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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The percentages of students in the 
different racial/ ethnic subgroups across 
jurisdictions who were at or above Proficient 
are presented in table 3.8 (grade 4) and 
table 3.9 (grade 8). The patterns seen in the 
grade 4 results are very similar to those 
found in the average score results. White 
students outperformed Black and Hispanic 
students in jurisdictions where sample sizes 
were large enough for reliable statistical 
comparisons. 



At grade 8, the percentage of White 
students in most states and jurisdictions at 
or above Prof dent was on average higher 
than the percentage of Black or Hispanic 
students in jurisdictions where a compari- 
son was possible. There were no statistically 
significant changes between 1996 and 2000 
in any state or jurisdiction in the percent- 
ages of eighth-graders in the different 
racial/ ethnic subgroups who were at or 
above Proficient. 
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Table 3.8 State Proficient Level Achievement Results by Race/Ethnicity, Grade 4 



State percentages of students at or 


above the Proficient level 


in science 


by race/ ethnicity for 


grade 4 public schools: 2000 


White 


Black 


Hispanic 


Asian/ 

Pacific Islander 


American 

Indian 


Nation 


37 


6 


10 


- 


17 


Alabama 


34 


5 


8 


**** 




Arizona 


34 


9 


7 


kkkk 


7 


Arkansas 


32 


3 


9 


kkkk 


22 


California ^ 


27 


4 


5 


19 


★★★★ 


Connecticut 


45 


4 


12 


kkkk 


kkkk 


Georgia 


39 


6 


12 


39 


kkkk 


Hawaii 


25 


8 


7 


16 


kkkk 


Idaho ^ 


35 


**** 


8 


kkkk 


kkkk 


Illinois ^ 


46 


7 


10 


kkkk 


kkkk 


Indiana ^ 


37 


9 


12 


kkkk 


kkkk 


Iowa ^ 


40 


■k-k-k-k 


16 


kkkk 


kkkk 


Kentucky 


32 


5 


15 


kkkk 


kkkk 


Louisiana 


31 


5 


17 


kkkk 


kkkk 


Maine ^ 


40 


kkkk 


16 


kkkk 


kkkk 


Maryland 


40 


6 


13 


44 


18 


Massachusetts 


50 


13 


11 


41 


kkkk 


Michigan ^ 


43 


6 


12 


★★★★ 


kkkk 


Minnesota ^ 


41 


7 


14 


11 


18 


Mississippi 


26 


2 


7 


**** 




Missouri 


42 


9 


20 


**** 


35 


Montana ^ 


41 


kkkk 


23 


**** 


19 


Nebraska 


31 


5 


12 


**** 




Nevada 


27 


4 


8 


21 


20 


New Mexico 


33 


9 


10 


kkkk 


6 


New York ^ 


40 


6 


9 


36 




North Carolina 


35 


6 


11 


kkkk 


10 


North Dakota 


41 


kkkk 


23 


kkkk 


13 


Ohio ^ 


38 


1 


17 


kkkk 


kkkk 


Oklahoma 


34 


9 


11 


kkkk 


22 


Oregon ^ 


32 


kkkk 


10 


kkkk 


26 


Rhode Island 


35 


5 


4 


18 


**** 


South Carolina 


34 


4 


11 


kkkk 


kkkk 


Tennessee 


34 


6 


9 


kkkk 


kkkk 


Texas 


39 


10 


12 


38 


kkkk 


Utah 


36 


kkkk 


13 


21 


16 


Vermont ^ 


40 


kkkk 


kkkk 




kkkk 


Virginia 


44 


12 


17 


58 


kkkk 


West Virginia 


26 


8 


12 




kkkk 


Wyoming 


37 


kkkk 


15 




22 


Other Jurisdictions 


American Samoa 




kkkk 


0 


▲ 


kkkk 


DDESS 


42 


15 


26 


25 


kkkk 


DoDDS 


41 


12 


23 


30 


24 


Guam 


7 


kkkk 


▲ 


4 


kkkk 


Virgin Islands 


**** 


4 


1 




kkkk 



**** Sample size is insufficient to permit a reliable estimate, t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

▲ Percentage is between 0.0 and 0.5. 

~ Special analyses raised concerns about the accuracy and precision of the National grade 4 Asian/Pacific Islander results in 2000. As a result, they are omitted from the 
body of this report. See appendix A for a more detailed discussion. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 3.9 State Proficient Leve\ Achievement Results by Race/Ethnicity, Grade 8 



State percentages of students at or above the Proficient level in science by race/ethnicity for grade 8 public 
schools: 1996 and 2000 

White Black Hispanic Asian/Pacific Islander American Indian 





1996 


2000 


1996 


2000 


1996 


2000 


1996 


2000 


1996 


2000 


Nation 


36 


40 


4 


6 


10 


11 


27 


36 


24 


14 


Alabama 


25 


31 


4 


4 


7 


7 


**** 


**** 


**★* 


**** 


Arizona 


33 


35 


7 


8 


8 


8 


**** 


**** 


6 


9 


Arkansas 


29 


30 


3 


2 


9 


8 


**** 


**** 


**** 


**** 


California 


33 


26 


5 


6 


6 


5 


27 


29 


**** 


**** 


Connecticut 


44 


45 


5 


6 


7 


11 


45 


44 


**** 


**** 


Georgia 


31 


36 


5 


6 


14 


13 


**** 


**** 


**** 


**** 


Hawaii 


23 


29 


9 


10 


7 


7 


15 


14 


**** 


**** 


Idaho ^ 


— 


42 


— 


**** 


— 


12 


— 


**** 





**** 


Illinois ^ 


— 


44 


— 


5 


— 


12 


— 


42 





**** 


Indiana ^ 


34 


40 


8 


6 


15 


12 


**** 


**** 


**** 


**** 


Kentucky 


25* 


32 


6 


7 


9 


**** 


**** 


**** 


**** 


**** 


Louisiana 


21* 


29 


3 


3 


7 


11 


**** 


**** 


**** 


**** 


Maine ^ 


43* 


38 


**** 


**** 


16 


**** 


**** 


**** 


**** 


**** 


Maryland 


38 


41 


5 


8 


8 


16 


38 


47 


**** 


**** 


Massachusetts 


41* 


49 


9 


12 


11 


12 


38 


46 


**** 


**** 


Michigan + 


39 


43 


6 


6 


14 


20 


**** 


**** 


**** 


**** 


Minnesota ^ 


40 


46 


9 


11 


13 


21 


30 


**** 


**** 


**** 


Mississippi 


22 


24 


3 


2 


3 


7 


**** 


**** 


•kidck 


**** 


Missouri 


34* 


42 


3 


7 


12 


19 


**** 


**** 


**** 


**** 


Montana ^ 


45 


49 


**** 


**** 


19 


29 


**** 


**** 


12 


25 


Nebraska 


38 


40 


7 


10 


16 


16 


**** 


**** 


**** 


**** 


Nevada 


— 


31 


— 


7 


— 


9 


— 


25 





14 


New Mexico 


36 


39 


**** 


**** 


9 


10 


**** 


**** 


8 


7 


New York ^ 


39 


44 


4 


8 


7 


11 


37 


29 


**** 


**** 


North Carolina 


33 


37 


6 


6 


8 


19 


**** 


36 


14 


**** 


North Dakota 


43 


44 


**** 


**** 


16 


21 


**** 


**** 


12 


12 


Ohio 


— 


45 


— 


11 


— 


30 





**** 





**** 


Oklahoma 


— 


32 


— 


7 


— 


10 


— 


**** 





19 


Oregon ^ 


34 


38 


**** 


8 


13 


10 


35 


38 


21 


22 


Rhode Island 


31 


34 


7 


6 


4 


9 


16 


26 


**** 


**** 


South Carolina 


29 


31 


4 


5 


7 


11 


**** 


**** 


**** 


**** 


Tennessee 


26 


31 


5 


6 


3 


13 


**** 


**** 


**** 


**** 


Texas 


38 


36 


6 


7 


8 


12 


34 


40 


**** 


**** 


Utah 


34 


38 


**** 


**** 


13 


15 


17 


32 


**** 


**** 


Vermont 


36* 


41 


**** 


**** 


16 


**** 


**** 


ifkick 


**** 


**** 


Virginia 


36 


39 


6 


9 


12 


18 


41 


49 


**** 


**** 


West Virginia 


22* 


28 


4 


7 


3 


**** 


**** 


**** 


**** 


**** 


Wyoming 

Other Jurisdictions 


37 


39 


**** 


**** 


14 


17 


**** 


**** 


8 


21 


American Samoa 


— 


**** 


— 


**** 


— 


0 


— 


3 





**** 


DDESS 


39 


48 


8 


13 


20 


31 


**** 


**** 


**** 


**** 


DoDDS 


42* 


50 


13 


16 


20 


28 


33 


37 


**** 


**** 


Guam 


23 


**** 


**** 


**** 


4 


2 


6 


7 


**** 


**** 



* Significantly different from 2000 if only one jurisdiction or the Nation is being examined. 

**** Sample size is insufficient to permit a reliable estimate. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Scale Score Differences Between 
Selected Subgroups by State 

Similar to results for the nation, changes in 
the score diflferences or “gaps’’ between 
male and female students were relatively 
small across states, and were not found to 
be significantly different across assessment 
years at grade 8. Also similar to the national 
data, the score gaps between male and 
female students are generally much smaller 
than those seen between racial/ethnic 
subgroups. None of the apparent changes 
in racial/ethnic score gaps across years at 
grade 8 were statistically significant. The 
gender and racial/ ethnic score gap results 
for jurisdictions are provided in appendix B. 

Free/Reduced-Price School Lunch 
Eligibility Results by State 

NAEP collects data on students’ eligibility 
for the federal free/ reduced-price school 
lunch program as an indicator of economic 
status in both the national and state (or 
jurisdiction) samples. Tables 3.10 and 3.11 
present the results by state and jurisdiction 
for grades 4 and 8, respectively. As previ- 
ously noted, comparison data for grade 4 
do not exist because the science assessment 
was only offered state-by-state at the 
eighth-grade level in 1996. 



At grade 4, in aU jurisdictions where 
sample sizes were large enough for reliable 
statistical comparisons, students who were 
not eligible for the free/reduced-price 
school lunch program outperformed 
students who were. A similar result was 
seen at grade 8. When data were compared 
across years, eighth-graders in five jurisdic- 
tions who were not eligible for the pro- 
gram had higher average scores in 2000 
than in 1996. They are: Louisiana, Missouri, 
Vermont, West Virginia, and DoDDS. 

The percentage of students at or above 
Proficient by free/ reduced-price school 
lunch ehgibihty in 2000 are presented for 
participating jurisdiction in tables 3.12 and 
3.13 for grades 4 and 8, respectively. There 
were higher percentages of eighth-graders 
who were not eHgible for the program at 
or above Proficient in 2000 than in 1996 in 
Lousiana, Missouri, and West Virginia. 
Additional data for these subgroups are 
included in appendix B. 



ERIC 



118 



CHAPTER 3 



SCIENCE REPORT CARO 



101 



Table 3.10 State Scale Score Results by Free/Reduced-Price School Lunch Eligibility, Grade 4 



State scale score results by student eligibility for free/reduced-price 


school lunch for 


grade 4 public schools: 2000 


Eligible 


Not eligible 


Information 
not available 


Nation 


129 


159 


160 


Alabama 


128 


159 


146 


Arizona 


125 


155 


136 


Arkansas 


131 


157 


**** 


California ^ 


115 


150 


137 


Connecticut 


135 


165 


144 


Georgia 


124 


159 


151 


Hawaii 


125 


147 


132 


Idaho ^ 


142 


159 


163 


Illinois ^ 


132 


163 


157 


Indiana ^ 


138 


162 


153 


Iowa ^ 


153 


163 


159 


Kentucky 


142 


161 


156 


Louisiana 


128 


159 


133 


Maine ^ 


150 


166 


161 


Maryland 


126 


158 


137 


Massachusetts 


139 


171 


155 


Michigan ^ 


134 


163 


131 


Minnesota ^ 


141 


163 


166 


Mississippi 


122 


153 


132 


Missouri 


141 


165 


145 


Montana ^ 


147 


167 


162 


Nebraska 


135 


159 


151 


Nevada 


128 


150 


137 


New Mexico 


126 


154 


146 


New York ^ 


133 


163 


158 


North Carolina 


131 


158 


155 


North Dakota 


150 


164 


159 


Ohio^ 


136 


164 


158 


Oklahoma 


144 


162 


149 


Oregon ^ 


136 


158 


147 


Rhode Island 


125 


162 


138 


South Carolina 


128 


157 


138 


Tennessee 


132 


159 


153 


Texas 


132 


160 


151 


Utah 


142 


160 


161 


Vermont ^ 


145 


165 


155 


Virginia 


138 


164 


163 


West Virginia 


143 


158 


152 


Wyoming 


148 


162 


155 


Other jurisdictions 


American Samoa 


51 






DDESS 


152 


160 


160 


DoDDS 


150 


158 


156 


Guam 


101 


121 




Virgin Islands 


115 




**** 



**** Sample size is insufficient to permit a reliable estimate. 

^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students vi/ith disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics. National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 3.11 State Scale Score Results by Free/Reduced-Price School Lunch Eligibility, Grade 8 



State scale score results by student eligibility for free/reduced-price school lunch for 



grade 8 public schools: 1996 and 2000 

Eligible 





1996 


2000 


Nation 


133* 


127 


Alabama 


121 


124 


Arizona ^ 


127 


127 


Arkansas 


128 


127 


California ^ 


120* 


113 


Connecticut 


127 


125 


Georgia 


124 


125 


Hawaii 


125 


119 


Idaho ^ 


— 


149 


Illinois ^ 


— 


126 


Indiana ^ 


136 


139 


Kentucky 


135 


139 


Louisiana 


121 


122 


Maine ^ 


152 


150 


Maryland 


122 


127 


Massachusetts 


133 


134 


Michigan ^ 


139 


134 


Minnesota ^ 


145 


141 


Mississippi 


121 


120 


Missouri 


138 


140 


Montana ^ 


150 


155 


Nebraska 


144 


142 


Nevada 


— 


126 


New Mexico 


130 


130 


New York ^ 


124 


132 


North Carolina 


128 


128 


North Dakota 


157* 


149 


Ohio 


— 


144 


Oklahoma 


— 


137 


Oregon ^ 


145 


138 


Rhode Island 


131 


130 


South Carolina 


126 


126 


Tennessee 


125 


129 


Texas 


130 


128 


Utah 


149* 


142 


Vermont ^ 


146 


144 


Virginia 


125 


130 


West Virginia 


138 


138 


Wyoming 


148 


147 


Other Jurisdictions 


American Samoa 


— 


72 


DDESS 


148 


153 


DoDDS 


146* 


155 


Guam 


101 


96 



Information 

Not eligible not available 



1996 


2000 


1996 


2000 


155* 


160 


154 


151 


150 


153 


151 


152 


155 


156 


144 


148 


152 


153 


155 


139 


152* 


145 


137 


135 


163 


163 


154 


147 


151 


155 


146 


145 


141 


142 


115* 


139 





164 


— 


155 





162 


— 


152 


158 


161 




149 


155* 


160 


142 




145 * 


155 


128 


133 


167 


163 


164 


155 


154 


158 


143 


138 


164 


168 


149 


164 


159 


164 


144 


152 


162 


165 


162 


164 


148 


149 


134 


138 


157 * 


164 


144 


153 


166 


170 


165 


168 


162 


162 


161 


161 





150 


— 


144 


151 


152 


143 


142 


159 


161 


153 


147 


156 


155 


144 


150 


165 


166 


155 


158 





166 


— 


151 





158 


— 


148 


159 


160 


151 


159 


157 


158 


125 


136 


149* 


155 




**** 


151 


155 


144 


147 


157 


156 


127 


137 


158 


159 


157 


158 


160* 


165 


157 


163 


157 


159 


150 


150 


152* 


158 


151 


151 


160 


161 


155 


159 




**** 




**** 


158 


163 


150 


158 


156* 


161 


156 


158 


125 


119 




104 



* Significantly different from 2000 if only one jurisdiction or the Nation is being examined. 

♦ Significantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on all jurisdictions that participated both years. 
**** Sample size is insufficient to permit a reliable estimate. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. . • 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table 3.12 State Proficientleve] Achievement Results by Free/Reduced-Price School Lunch Eligibility, Grade 4 



State percentages of students at or above the Projiaent level in science by student eligibility for 
free/reduced-price school lunch program for grade 4 public schools: 2000 



Eligible 



Nation ll 

Alabama g 

Arizona 8 

Arkansas 13 

California ^ 4 

Connecticut 12 

Georgia 7 

Hawaii 8 

Idaho ^ 19 

Illinois ^ 12 

Indiana ^ 14 

Iowa ^ 26 

Kentucky 17 

Louisiana iQ 

Maine ^ 23 

Maryland 7 

Massachusetts 16 

Michigan ^ 15 

Minnesota ^ 17 

Mississippi 6 

Missouri 19 

Montana ^ 23 

Nebraska H 

Nevada 8 

New Mexico 9 

New York ^ 11 

North Carolina 9 

North Dakota 26 

Ohlo^ 12 

Oklahoma 17 

Oregon ^ 15 

Rhode Island 8 

South Carolina 9 

Tennessee 12 

Texas 9 

Utah 19 

Vermont ^ 22 

Virginia 12 

West Virginia 17 

Wyoming 21 





Information 


)t eligible 


not available 


37 


39 


36 


23 


34 


19 


35 


irkirk 


26 


16 


44 


26 


37 


27 


23 


11 


36 


41 


42 


42 


40 


31 


41 


36 


38 


35 


36 


13 


46 


36 


36 


19 


53 


37 


43 


12 


41 


49 


28 


12 


44 


29 


46 


41 


35 


29 


26 


13 


30 


26 


39 


36 


34 


29 


43 


38 


43 


32 


39 


23 


35 


30 


38 


19 


34 


16 


36 


36 


37 


30 


37 


40 


45 


34 


42 


43 


33 


26 


38 


30 



Other Jurisdictions 






American Samoa 


A 


**** 


DDESS 


23 


35 


DoDDS 


22 


33 


Guam 


2 


6 


Virgin Islands 


3 





**** 

32 

31 

**** 



**** Sample size is insufficient to permit a reliable estimate. 

Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

A Percentage Is between 0.0 and 0.5. 

performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 3.13 State Proficient \.eye\ Achievement Results by Free/Reduced-Price School Lunch Eligibility, 



State percentages of students at or above the Proficient 
free/reduced-price school lunch program for grade 8 



level in science by student eligibility for 
public schools: 1996 and 2000 



Eligible 



Not eligible 



Information 
not available 





1996 


2000 


1996 


2000 


1996 


2000 


Nation 


14 


12 


32* 


39 


34 


31 


Alabama 


7 


9 


24 


31 


33 


31 


Arizona ^ 


9 


10 


31 


31 


18 


25 


Arkansas 


10 


12 


28 


30 


30 


22 


California ^ 


6 


4 


31 


23 


15 


17 


Connecticut 


10 


7 


43 


43 


38 


29 


Georgia 


6 


9 


29 


33 


25 


23 


Hawaii 


9 


7 


18 


20 


5* 


20 


Idaho ^ 


— 


27 


— 


44 


— 


36 


Illinois ^ 


— 


10 


— 


40 


— 


28 


Indiana ^ 


12 


16 


35 


41 


**** 


28 


Kentucky 


11 


16 


31* 


38 


16 


**** 


Louisiana 


7 


8 


20* 


32 


16 


13 


Maine ^ 


27 


25 


46 


41 


41 


28 


Maryland 


8 


9 


32 


37 


16 


17 


Massachusetts 


13 


14 


44 


49 


29 


46 


Michigan ^ 


17 


16 


38 


44 


26 


32 


Minnesota ^ 


22 


21 


40 


47 


42 


45 


Mississippi 


5 


6 


22 


24 


9 


17 


Missouri 


15 


18 


34* 


44 


25 


32 


Montana ^ 


25 


34 


46 


51 


43 


48 


Nebraska 


20 


21 


40 


41 


38 


44 


Nevada 


— 


10 


— 


28 


— 


17 


New Mexico 


10 


11 


28 


29 


19 


24 


New York ^ 


10 


14 


37 


41 


36 


28 


North Carolina 


7 


9 


33 


34 


17 


35 


North Dakota 


33 


26 


44 


47 


33 


36 


Ohio 


— 


22 


— 


46 


— 


33 


Oklahoma 


— 


16 


— 


33 


— 


27 


Oregon ^ 


20 


17 


37 


39 


30 


38 


Rhode Island 


10 


10 


32 


36 


10 


14 


South Carolina 


7 


8 


26 


31 


**** 


ieieieie 


Tennessee 


9 


11 


28 


33 


23 


26 


Texas 


9 


9 


34 


33 


14 


21 


Utah 


25 


23 


34 


38 


32 


37 


Vermont ^ 


22 


22 


38* 


44 


30* 


43 


Virginia 


6 


11 


34 


37 


27 


29 


West Virginia 


12 


14 


26* 


35 


23 


25 


Wyoming 


22 


24 


37 


40 


32 


33 


Other Jurisdictions 


American Samoa 


— 


2 


— 




— 




DDESS 


20 


29 


32 


40 


25 


35 


DoDDS 


20* 


33 


33* 


39 


31 


37 


Guam 


▲ 


3 


9 


7 




5 



O 
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* Significantly different from 2000 if only one jurisdiction or the Nation is being examined. 

* Significantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on all jurisdictions that participated 
both years. 

t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. ▲ Percentage is between 0.0 and 0.5. 

NOTE: Comparative performance results may be affected by changes in exclusion rates for students with disabilities and limited English proficient students in the NAEP samples. 
DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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4 Becoming a More Inclusive 

National Assessment 



Legislation at the federal level now^ mandates the inclusion 
ofaU students in large-scale academic assessments.^ As a 
consequence, most states have assessment programs that must 
make provisions for special-needs students — those w^ith 
disabihties or limited EngHsh proficient students — that 
include the allov^^ance of testing accommodations w^hen 
appropriate. Assessing as representative a sample of the 
nation’s students as possible is particularly important for 
NAEP’s mission to serve as a key indicator of the 
academic achievement of the nation’s students. This 
mission can be satisfactorily accomplished only if the 
assessment results include data gathered from aU 
groups of students, including those classified as 
having special needs. 

Although the intent of NAEP has consistently 
been to include special-needs students in its 
assessments to the fullest degree possible, the 
implementation of the assessment has always resulted 
in some exclusion of students who could not be 
assessed meaningfully without accommodations. 
Participating schools have been permitted to exclude 
certain students who have been classified as having a 
disability under the Individuals with Disabihties Education 
Act, based upon their Individuahzed Education Programs 
(lEP) and Section 504 of the Rehabilitation Act of 1973. 




How would the 
NAEP results 
differ if 

accommodations 
were permitted 
for special-needs 
students? 




Chapter 

Contents 



Two sets of ! 
I NAEP 2000 I 
I Science Results I 

I I 

I National Results 

! 

I 

National Results 
by Gender 

National Results 
by Race/Ethnicity 

State Results 



^ Public Law 105-17. (1997). Individuals with Disabilities Education Act (IDEA). See 
also:TideVI of the Civil Rights Act, Equal Educational Opportunities Act, Section 
504 of the Rehabilitation Act. 
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Similarly, schools have been permitted to 
exclude some students they identify as being 
hmited-Enghsh proficient. Exclusion 
decisions are made in accordance with 
exphcit criteria provided by the NAEP 
program. 

In order to move the NAEP assessments 
toward more inclusive samples, the NAEP 
program began to explore the use of 
accommodations with special-needs stu- 
dents during the 1996 science assessment. 
An additional impetus for this change was 
the attempt to keep NAEP consistent with 
state and district testing policies that 
increasingly offered accommodations so 
that more special-needs students could be 
assessed. In 1996, the national NAEP 
sample was spht so that some of the schools 
sampled were permitted to provide accom- 
modations to special-needs students and 
the other schools were not. This sample 
design made it possible to study the effects 
on NAEP results of including special-needs 
students in the assessments under alternate 
testing conditions. Technical research 
papers have been published with the results 
of these comparisons.^ Based on the out- 
comes of these analyses, the 1998 results of 
those NAEP assessments that used new test 
frameworks (writing and civics), and hence 
also began new trend fines, were reported 
with the inclusion of data from accommo- 
dated special-needs students. 



The results presented in the NAEP 1996 
science report card included the perfor- 
mance of those students with disabilities 
(SD) or limited English proficient students 
(LEP) who were assessed without the 
possibility of accommodations. They did 
not include results on the performance of 
students for whom accommodations were 
permitted. However, in both the 1996 and 
2000 science assessments, NAEP used the 
split-sample design so that changes in 
students’ science achievement could be 
reported across the two assessment years 
and, at the same time, the program could 
continue to examine the effects of includ- 
ing students assessed with accommodations. 

Two Sets of 2000 NAEP 
Science Results 

This report card is the first to display two 
different sets of NAEP science results based 
on the split-sample design: 1) those that 
reflect the performance of regular and 
special-needs students when accommoda- 
tions were not permitted, and 2) those that 
reflect the performance of regular and 
special-needs students — both those who 
were accommodated and those who were 
tested without accommodations — when 
accommodations were permitted. It should 
be noted that accommodated students 
make up a small proportion of the total 
weighted number of students assessed (see 
table A.9, in appendix A for details). Mak- 



2 Olson, J. F, & Goldstein, A, A. (1997), The inclusion of students with disabilities and limited English proficient students in 
large-scale assessments: A summary of recent progress. (NCES Publication No, 97-482), Washington, DC: U,S, Depart- 
ment of Education, Office of Educational Research and Improvement, National Center for Education Statistics, 

Mazzeo,J,, Carlson, J. E,,Voelkl, K, E,, & Lutkus,A, D, (1999), Increasing the participation of special needs students in 
NAEP: A report on 1996 research activities. (NCES Publication No, 2000-473). Washington, DC: U.S. Department 
of Education, Office of Educational Research and Improvement, National Center for Education Statistics, 

Lutkus,A, D,,& Mazzeo,J, Including special-needs students in the NAEP 1998 reading assessment: Part I, comparison of 
overall results with and without accommodations. Washington, DC: U.S, Department of Education, Institute of Educa- 
tion Sciences, National Center for Education Statistics (forthcoming). 

Lutkus, A. D. Including special-needs students in the NAEP 1998 reading assessment: Part If results for students with 
disabilities and limited English proficient students. Washington, DC: US. Department of Education, Institute of 
Education Sciences, National Center for Education Statistics (forthcoming). 
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ing accommodations available may change 
the overall assessment results in subtle and 
different >vays. For example, when accom- 
modations are permitted, there may be 
some occurrences of students being ac- 
commodated who might have taken the 
test under standard conditions if accommo- 
dations were not permitted. This could lead 
to an overall increase in the average assess- 
ment results, if accommodations were to 
increase special-needs students’ perfor- 
mance. Conversely, when accommodations 
are permitted, many special-needs students 
who could not have been tested without 
accommodations could be included in the 
sample. Assuming that these are generally 
lower-performing students, their inclusion 
in the sample — even with accommoda- 
tions — could result in an overall lower 
average score. 

Chapters 1, 2, 3, and 5 of this report are 
based on the first set of results (i.e., no 
accommodations offered). This chapter 
presents an overview of the second set of 
results — ^those that include students who 
were provided with accommodations 
during the assessment administration. By 
including these results, the NAEP program 
begins a phased transition toward a more 
inclusive reporting sample. Future assess- 
ment results will be based solely on student 
and school samples in which accommoda- 
tions are permitted. 

The two sets of results presented in this 
chapter were obtained by administering the 
assessment to a nationally representative 
sample of students and schools. In one part 
of the schools sampled, no accommoda- 
tions were permitted: all students were 
assessed under the same conditions that 
were the basis for reporting results from the 
1996 NAEP science assessments. In another 
part of the schools sampled, accommoda- 







tions were permitted for SD and LEP 
students who normally receive accommo- 
dations in their district or state assessment 
programs. Most accommodations that 
schools routinely provide for their own 
testing programs were permitted. Such 
permitted accommodations included, but 
were not limited to the following: 

• one-on-one testing, 

• bilingual dictionary, 

• large print book, 

• small-group testing, 

• extended time, 

• oral reading of directions, and 

• use of an aide for transcribing responses. 

(See appendix A, table A. 11, for greater 
detail on the numbers and percentages of 
students accommodated by accommoda- 
tion type in the 1996 and 2000 science 
assessments.) 

Figure 4.1 provides a visual representa- 
tion of how the two sets of results were 
based on the two samples in 1996 and 
2000. Included in both sets of results 
(accommodations not permitted and 
accommodations permitted) are those 
students from both samples of schools who 
were not identified as either SD or LEP. In 
addition, the first set of results (accommo- 
dations not permitted) includes SD and 
LEP students from the sample of schools 
where accommodations were not permit- 
ted (see middle portion of figure 4.1). This 
is the set of results that allowed for com- 
parisons to 1996 and that are presented in 
the other chapters of this report. 

The second set of results, accommoda- 
tions permitted (see bottom portion of 
figure 4.1), includes SD and LEP students 
firom the sample of schools where accom- 
modations were permitted. This is the set 
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Figure 4.1 Split-Sample Design 



The two sets of NAEP results based 
on a split-sample design 



Sample with no 
accommodations permitted 



Sample with 

accommodations permitted 



Non-SD/LEP Non-SD/LEP 

students students 



Split-sample design 

The national sample was split. In part of the 
schools, accommodations were not permitted 
for students with disabilities (SD) and limited 
English proficient students (LEP). In the other 
schools, accommodations were permitted for 
SD and LEP students who routinely received 
them in their school assessments. 



SD/lEP SD/LEP 

students students 



Sample with no Sample with 

accommodations permitted accommodations permitted 



r 

Non-SD/LEP 

students 



.<tn/i FP 




SD/LEP 

students 



Accommodations-not-permitted results 

The accommodations-not-permitted results 
include the performance of students from both 
samples who were not classified as SD or LEP 
and the performance of SD and LEP students 
from the sample in which no accommodations 
were permitted. 



Sample with no Sample with 

accommodations permitted accommodations permitted 





Accommodations-permitted results 

The accommodations-permitted results also 
include the performance of students from both 
samples who were not classified as SD or LEP; 
however, the SD and LEP students whose 
performance is included in this set of 
results were from the sample in which 
accommodations were permitted. Since 
students who required testing accommodations 
could be assessed and represented in the 
overall results, it was anticipated that these 
results would include more special-needs 
students and reflect a mor^ inclusive sample. 
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of results that form the new, more inclusive 
baseline for future reporting of trend 
comparisons for the NAEP science 
assessment. 

In the NAEP 2000 national sample 
where accommodations were not 
permitted, 14 percent of fourth-graders, 

14 percent of eighth-graders, and 9 percent 
of twelfth-graders, were identified by their 
schools as having special needs (i.e., either 
as SD or LEP students). In the other 
national sample where accommodations 
were offered, 16 percent of fourth-graders, 
13 percent of eighth-graders, and 9 percent 
of twelfth-graders were identified as having 
special needs. In the sample where accom- 
modations were not permitted, 48 percent 
of the special-needs students at fourth and 
twelfth grade, and 49 percent at eighth 
grade (between 4 and 7 percent of all 
students — see appendix A, table A. 7) were 
excluded from NAEP testing by their 
schools. In the sample where accommoda- 
tions were offered, 28 percent of the 
special-needs students at each of the three 
grade levels were excluded from the assess- 
ment (between 2 and 4 percent of the total 
sample). 

The focus of this chapter is a 
comparison of data from the two sets of 
results: 1) accommodations not permitted, 
and 2) accommodations permitted. Because 
the split-sample design was used in both 
1996 and 2000 for the NAEP national 
science assessment, both sets of results are 
presented for both years. The spHt-sample 
design was first used in the NAEP state 
science assessment in 2000. Overall results 



are provided for the nation and for partici- 
pating states and other jurisdictions. In 
addition, national results are presented by 
gender and by race/ethnicity. These results 
are discussed in terms of statistically signifi- 
cant differences between the two sets of 
results in each year, changes between 
assessment years, and differences between 
subgroups of students within each set of 
results. Throughout this chapter, the assess- 
ment results that include SD and LEP 
students for whom accommodations were 
not permitted will be referred to as the 
“accommodations-not-permitted” results. 
The set of results that includes SD and LEP 
students for whom accommodations were 
permitted will be referred to as the “ac- 
commodations-permitted” results. 

Results for the Nation 
Accommodations Not Permitted and 
Accommodations Permitted 

Table 4.1 displays the average science scale 
scores for the nation in 1996 and 2000 for 
two sets of results: 1) accommodations not 
permitted, and 2) accommodations permit- 
ted. At grade 4, the accommodations- 
permitted average score in 2000 v^s two 
points lower than the accommodations- 
not-permitted average score. The small 
difference between the two sets of results in 
1996 was not statistically significant. At 
grades 8 and 12 the apparent differences 
between the two average scores in either 
1996 or 2000 were not found to be statisti- 
cally significant. The decline in the average 
twelfth-grade score between 1996 and 
2000 is evident in both sets of results. 
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Table 4.1 Comparison of Two Sets of National Scale Score Results 



National average science scale scores by type of results, grades 4, 8, and 12; 1996 and 2000 





Accommodations not permitted 


Accommodations permitted 


Grade 4 


1996 


150 


149 


2000 


150 


148 1 


Grade 8 


1996 


150 


150 


2000 


151 


151 


Grade 12 


1996 


150* 


150* 


2000 


147 


146 



* Significantly different from 2000. 

^ Significantly different from the result where accommodations were not permitted. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



As noted in the introduction to this 
chapter, NAEP has always sought to in- 
clude special-needs students proportionate 
to their representation in the U.S. popula- 
tion. Offering accommodations tends to 
reduce exclusion rates for special-needs 
students and therefore allows NAEP to 
offer a fairer and more accurate picture of 
the status of American education. Because 
special-needs students are typically classi- 
fied as eligible for special educational 
services after having shown some difficulty 
in the regular learning environment, the 
academic achievement of special-needs 
students might be expected to be lower 
than that of students without such needs. 
This only appeared to be the case in the 
observed difference between the two sets 
of grade 4 science results in 2000, where 
the accommodations-permitted results, 
which included sHghtly more special-needs 



students because of the availability of 
accommodations, were lower than the 
accommodations-not-permitted results. It 
is important to examine the percentages of 
students attaining the NAEP achievement 
levels, however, to see if there were higher 
percentages at the lower achievement levels 
(i.e., below Basic and Basic), when students 
were assessed with accommodations. 

Table 4.2 shows the percentages of 
students attaining each of the achievement 
levels. The percentages are similar across the 
two sets of 1996 results for grades 8 and 12: 
apparent differences between the accom- 
modations-not-permitted and the accom- 
modations-permitted results were not 
found to be significantly different. At grade 
4, however, the percentage of students 
below Basic in both years was higher when 
accommodations were permitted than 
when they were not. 
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Table 4.2 Comparison of Two Sets of National Achievement-Level Results 



Percentage of students within each science achievement-level range and at or above 
achievement levels by type of results, grades 4, 8, and 12: 1996 and 2000 



[ j At or above 



Grade 4 

1996: Accommodations were 


Below Bssic 


At Basic 


At Proficient 


[[{Advanced | 




1 Proficient 


not permitted 


33 


38 


26 


3 


67 


29 


permitted 

2000: Accommodations were 


35^ 


36 1 


25 


4 


65^ 


29 


not permitted 


34 


37 


26 


4 


66 


29 


permitted 

Grade 8 

1996: Accommodations were 


36 1 


36 


25 


3 


64 1 


29 


not permitted 


39 


32* 


26 


3 


61 


29* 


permitted 

2000: Accommodations were 


39 

1 


31* 


26 


3* 


61 


29 


not permitted 


39 


29 


28 


4 


61 


32 


permitted 

Grade 12 

1996: Accommodations were 


39 


29 


27 


4 


61 


32 


not permitted 


43* 


36 


19 


3 


57* 


21 


permitted 

2000: Accommodations were 


43* 


35 


19* 


3 


57* 


21* 


not permitted 


47 


34 


16 


2 


53 


18 


permitted 


48 


34 


16 


2 


52 


18 



* Significantly different from 2000. 

t Significantly different from the result where accommodations were not permitted. 

NOTE: Percentages within each science achievement-level range may not add to 100 or to the exact percentages at or above achievement levels due to 
rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



National Results by Gender 
Accommodations Not Permitted and 
Accommodations Permitted 

The average science scale scores by gender 
for both sets of results in 1996 and 2000 
are provided in table B.67 in appendix B. 

In 2000, male students at grade 4 had 
higher science scores when accommoda- 
tions were not permitted than when 
accommodations were permitted. 

At grades 4 and 8, male students outper- 
formed female students in 2000 regardless 
of whether or not accommodations were 
permitted. At grade 12, the apparent 
diflference in scores between male and 



female students was not statistically signifi- 
cant in either set of results. 

There was no variation in the two sets of 
results with respect to diflferences in the 
performance of male and female students 
between 1996 and 2000. In both sets of 
results, male students had higher average 
scores in 2000 than in 1996 at grade 8, and 
lower average scores in 2000 at grade 12. 
The performance among female students 
also remained stable between 1996 and 
2000 at all three grades, with no statistically 
significant diflferences observed over time 
in either set of results. 
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The percentages of male and female 
students attaining the Basic, Proficient, and 
Advanced levels are provided in table B.68 
in appendix B. Comparing the two sets of 
results both in 1996 and 2000, a higher 
percentage of fourth-grade males were 
below Basic when accommodations were 
permitted in 2000 than when they were 
not. No statistically significant differences 
were found in the percentages of students 
attaining each of the achievement levels at 
grades 8 or 12. 

National Results by 
Race/Ethnicity 

Accommodations Not Permitted and 
Accommodations Permitted 

NAEP assessments across academic subjects 
have typically reported large score differ- 
ences between different racial and ethnic 
subgroups. If students with disabilities (SD) 
or hmited Engfish proficient (LEP) students 
are overrepresented in a particular racial or 
ethnic group, that group’s assessment scores 
may decrease. Table B.69 in appendix B 
provides the average science scale scores for 
each of the race/ethnicity categories for 
the two sets of results in 1996 and in 2000. 
There were no statistically significant 
differences observed between the average 
scores when accommodations were not 
permitted and when accommodations were 
permitted for any of the race/ethnicity 
categories in either 1996 or 2000. 

As noted in chapter 3, a pattern of 
performance differences by race /ethnicity 
can be seen in the accommodations-not- 
permitted results in 2000. Similar patterns 
were observed in the accommodations- 
permitted results with three exceptions. 
American Indian eighth-graders scored 
higher than Hispanic eighth-graders when 
accommodations were permitted, while the 
apparent difference was not statistically 
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significant when accommodations were not 
permitted. Hispanic twelfth-graders scored 
higher than Black twelfth-graders when 
accommodations were permitted but not 
significantly different from each other 
when accommodations were not permit- 
ted. Finally, the difference in average 
science scores between Asian/Pacific 
Islander and American Indian twelfth- 
graders was not significantly different when 
accommodations were permitted, while 
Asian/Pacific Islander students outper- 
formed American Indian students when 
accommodations were not permitted. 

At grade 8, American Indian students 
scored lower in 2000 than in 1996 when 
accommodations were not permitted, 
while the apparent decrease was not statis- 
tically significant when accommodations 
were permitted. 

The percentages of students in each 
race/ ethnicity category who attained the 
Basic, Proficient, and Advanced levels are 
provided in table B.70 in appendix B. No 
statistically significant differences were 
found between the accommodations-not- 
permitted results and the accommodations- 
permitted results for the percentages of 
students attaining any of the achievement 
levels at any of the grade levels in 1996 
and 2000. 

state Results 

Accommodations Not Permitted and 
Accommodations Permitted 

While the split-sample design was used for 
both the 1996 and 2000 national assess- 
ments, it was used for the first time in the 
state assessment of science in 2000. The 
two sets of average scale scores for the 
jurisdictions that participated in 2000 are 
presented in tables 4.3 and 4.4 for grades 4 
and 8, respectively. As with the presentation 
of results for jurisdictions in previous 
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chapters, two types of statistical tests are 
indicated in these tables — one that involves 
a multiple-comparison procedure based on 
all jurisdictions that participated, and one 
that examines each jurisdiction separately. 
The following discussion of differences 
between the accommodations-not-permit- 
ted results and the accommodations- 
permitted results is based solely on the 
multiple-comparison procedure. 

None of the apparent differences be- 
tween the accommodations-not-permitted 
results and the accommodations-permitted 
results for either grade 4 or grade 8 were 
found to be statistically significant. 

Figures 4.3 and 4.4 show comparisons of 
scale scores across states when accommoda- 
tions were permitted for fourth- and 
eighth-grade students, respectively. Six 
states were included among the highest- 
performing jurisdictions at grade 4: Iowa, 



Maine, Massachusetts, Montana, North 
Dakota and Vermont. These states were also 
included among the highest-performing 
jurisdictions when accommodations were 
not permitted. At grade 8, a cluster of high- 
performing jurisdictions when accommo- 
dations were permitted included Depart- 
ment of Defense Dependents Schools 
(Overseas), Idaho, Maine, Massachusetts, 
Michigan, Minnesota, Nebraska, North 
Dakota, Ohio, and Vermont. This cluster of 
10 states was outperformed only by 
Montana. Most of these states were also 
among the higher-performing jurisdictions 
when accommodations were not permit- 
ted. Michigan had lower average scores 
than Massachusetts, Vermont, and North 
Dakota, and scores in Nebraska were lower 
than in Vermont and North Dakota when 
accommodations were not permitted. A 
listing of these jurisdictions by type of 
results is presented in figure 4.2. 



Figure 4.2 Highest Performing Jurisdictions by Type of Results 



States with highest average science scale scores that did not differ from each other by type of 
results for grades 4 and 8: 2000 



Grade 4 Grade 8 



Accommodations 
not permitted 


Accommodations 

permitted 


Accommodations 
not permitted 


Accommodations 

permitted 


Iowa 


Iowa 


Idaho 


Idaho 


Maine 


Maine 


Maine 


Maine 


Massachusetts 


Massachusetts 


Massachusetts 


Massachusetts 


Montana 


Montana 


Minnesota 


Michigan 


North Dakota 


North Dakota 


*Montana 


Minnesota 


Vermont 


Vermont 


North Dakota 


* Montana 






Ohio 


Nebraska 






Vermont 


North Dakota 






DDESS 


Ohio 






DoDDS 


Vermont 








DoDDS 



‘Average science scores in Montana were higher than the other states listed for grade 8. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 4.3 Comparison of Two Sets of State Scale Score Results, Grade 4 



State average science scale scores by type of results for grade 4 public schools: 2000 





Accommodations 


Accommodations 




not permitted 


permitted 


Nation 


148 


147 


Alabama 


143 


143 


Arizona 


141 


140 


Arkansas 


144 


145 


California ^ 


131 


129 


Connecticut 


156 


156 


Georgia 


143 


142 


Hawaii 


136 


136 


Idaho t 


153 


152 


Illinois^ 


151 


150 


Indiana ^ 


155 


154 


Iowa ^ 


160 


159 


Kentucky 


152 


152 


Louisiana 


139 


139 


Maine ^ 


161 


161 


Maryland 


146 


145 


Massachusetts 


162 


161 


Michigan ^ 


154 


152 


Minnesota ^ 


157 


157 


Mississippi 


133 


133 


Missouri 


156 


157 


Montana ^ 


160 


160 


Nebraska 


150 


150 


Nevada 


142 


142 


New Mexico 


138 


140 


New York ^ 


149 


148 


North Carolina 


148 


147 


North Dakota 


160 


160 


Ohiot 


154 


155 


Oklahoma 


152 


151 


Oregon ^ 


150 


148 


Rhode Island 


148 


148 


South Carolina 


141 


140 


Tennessee 


147 


145 


Texas 


147 


145 


Utah 


155 


154 


Vermont ^ 


159 


160 


Virginia 


156 


155 


West Virginia 


150 


149 


Wyoming 


158 


156 


Other Jurisdictions 


American Samoa 


51 


54 


DDESS 


157 


157 


DoDDS 


156 


155 


Guam 


no 


114 


Virgin Islands 


116 


116 



t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table 4.4 Comparison of Two Sets of State Scale Score Results, Grade 8 I 


State average science scale 


scores by type 


of results for grade 8 public schools: 2000 




Accommodations 


Accommodations 




not permitted 


permitted 


Nation 


149 


149 


Alabama 


141 


143 


Arizona ^ 


146 


145 


Arkansas 


143 


142 


California ^ 


132 


129 


Connecticut 


154 


153 


Georgia 


144 


142 


Hawaii 


132 


130 


Idaho ^ 


159 


158 


Illinois ^ 


150 


148 


Indiana ^ 


156 


154 


Kentucky 


152 


150 


Louisiana 


136 


134 


Maine ^ 


160 


158 


Maryland 


149 


146 


Massachusetts 


161 


158 


Michigan ^ 


156 


155 


Minnesota ^ 


160 


159 


Mississippi 


134 


134 


Missouri 


156 


154 


Montana ^ 


165 


164 


Nebraska 


157 


158 


Nevada 


143 


141 


New Mexico 


140 


139 


New York ^ 


149 


145 


North Carolina 


147 


145 


North Dakota 


161 


159 


Ohio 


161 


159 


Oklahoma 


149 


149 


Oregon ^ 


154 


154 


Rhode Island 


150 


148 


South Carolina 


142 


140 


Tennessee 


146 


145 


Texas 


144 


143 


Utah 


155 


154 


Vermont ^ 


161 


159 


Virginia 


152 


151 


West Virginia 


150 


146* 


Wyoming 


158 


156 


Other Jurisdictions 


American Samoa 


72 


74 


DDESS 


159 


155 


DoDDS 


159 


159 


Guam 


114 


114 



t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

* Significantly different from the result where accommodations were not permitted when examining only one jurisdiction. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Figure 4.3 Cross-State Scale Score Comparisons for Accommodations-Permitted Results, Grade 4 



Comparisons of average science scale scores for grade 4 public schools: 2000 sample where 
accommodations were permitted 

Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity surrounding 
a jurisdiction's abbreviation to the key below to determine whether the average science scale score of this jurisdiction is higher than, the same as, or lower 
than the jurisdiction in the column heading. For example, in the column under Indiana, Indiana's score was lower than Maine, Massachusetts, Vermont, North 
Dakota, Montana, and Iowa, about the same as all the states from Minnesota through Nebraska, and higher than the remaining states down the column. 
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□ 

□ 

□ 



Jurisdiction has statistically significantly higher average scale 
score than the jurisdiction listed at the top of the figure. 

No statistically significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statistically significantly lower average scale 
score than the jurisdiction listed at the top of the figure. 



er|c 



CHAPTER 4 




• SCIENCE REPORT CARO 



The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Significance Is determined by an 
application of a multiple comparison procedure (see appendix A). 

■►•►Indicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS; Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Differences between states and jurisdictions may be partially 
explained by other factors not Included in this figure. 

SOURCE: National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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Figure 4.4 Cross-State Scale Score Comparisons for Accommodations-Permitted Results, Grade 8 



Comparisons of average science scale scores for grade 8 public schools: 2000 sample where 
accommodations were permitted 

Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction’s abbreviation to the key below to determine whether the average science scale score of this jurisdiction is higher than, the 
same as, or lower than the jurisdiction in the column heading. For example, in the column under Ohio, Ohio’s score was lower than Montana, about the 
same as'all the states from North Dakota through Michigan, and higher than the remaining states down the column. 
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! Jurisdiction has statistically significantly higher average scale 
I score than the jurisdiction listed at the top of the figure. 

I No statistically significant difference detected from the jurisdiction 
I listed at the top of the figure. 

I Jurisdiction has statistically significantly lower average scale 
I score than the jurisdiction listed at the top of the figure. 



BEST COPY AVAILABLE 



The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Significance is determined by an 
application of a multiple comparison procedure (see appendix A). 

Indicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS: Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Differences between states and jurisdictions may be partially 
explained by other factors not included in this figure. 

SOURCE: National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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Tables 4.5 and 4.6 show the percentages 
of students in each jurisdiction who were 
at or above the Proficient level when ac- 
commodations were not permitted and 
when accommodations were permitted. 
Again, no statistically significant differences 
were observed between the two sets of 
results at both grades 4 and 8. 

Figures 4.5 and 4.6 indicate whether 
differences in the percentages of students at 
or above Prof dent between pairs of partici- 
pating jurisdictions were statistically signifi- 
cant when accommodations were permit- 
ted. At grade 4, the cluster of four states 
with the highest percentage at or above the 



Proficient level included Maine, Massachu- 
setts, Montana, and Vermont. The same four 
states were also included among the juris- 
dictions clustered at the top when accom- 
modations were not permitted (see chapter 
2). At grade 8, Massachusetts, Minnesota, 
Montana, North Dakota, Ohio, andVer- 
mont, had the highest percentages of 
students at or above Prof dent when accom- 
modations were permitted. Only four of 
these six states were among those with the 
highest percentage at or above the Proficient 
level (Massachusetts, Minnesota, Montana, 
and Ohio), in the accommodations-not- 
permitted results for grade 8. 
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Table 4.5 Comparisons of Two Sets of State Prof/c/enf Level Results, Grade 4 



Percentage of students at or above the Proficient level in science by state and type of results for 
grade 4 public schools: 2000 





Accommodations 


Accommodations 




not permitted 


permitted 


Nation 


28 


27 


Alabama 


22 


22 


Arizona 


22 


22 


Arkansas 


24 


23 


California ^ 


14 


13 


Connecticut 


35 


35 


Georgia 


23 


23 


Hawaii 


16 


16 


Idaho ^ 


30 


29 


Illinois ^ 


31 


31 


Indiana ^ 


32 


32 


Iowa ^ 


37 


36 


Kentucky 


29 


28 


Louisiana 


19 


18 


Maine ^ 


38 


37 


Maryland 


26 


24 


Massachusetts 


43 


42 


Michigan ^ 


33 


32 


Minnesota ^ 


35 


34 


Mississippi 


14 


13 


Missouri 


35 


34 


Montana ^ 


37 


36 


Nebraska 


26 


26 


Nevada 


19 


19 


New Mexico 


18 


17 


New York ^ 


26 


24 


North Carolina 


24 


23 


North Dakota 


38 


36 


Ohio^ 


31 


31 


Oklahoma 


26 


26 


Oregon ^ 


28 


27 


Rhode Island 


27 


25 


South Carolina 


21 


20 


Tennessee 


26 


24 


Texas 


24 


23 


Utah 


32 


31 


Vermont ^ 


39 


38 


Virginia 


33 


32 


West Virginia 


25 


24 


Wyoming 


33 


31 


Other Jurisdictions 


American Samoa 


A 


A 


DDESS 


29 


30 


DoDDS 


30 


30 


Guam 


4 


4 


Virgin Islands 


4 


4 




t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

A Percentage is between 0.0 and 0.5. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: National results are based on the national sample, not on aggregated state assessment samples. 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 



137 



CHAPTER 4 



SCIENCE REPORT CARO 



121 




Table 4.6 Comparisons of Two Sets of State Proficientleve\ Results, Grade 8 



Percentage of students at or above the Proficient level in science by state and type of results for 
grade 8 public schools: 2000 





Accommodations 


Accommodations 




not permitted 


permitted 


Nation 


30 


30 


Alabama 


22 


23 


Arizona ^ 


24 


23 


Arkansas 


23 


22 


California ^ 


15 


14 


Connecticut 


35 


35 


Georgia 


23 


23 


Hawaii 


15 


14 


Idaho t 


38 


37 


Illinois ^ 


30 


29 


Indiana ^ 


35 


33 


Kentucky 


29 


28 


Louisiana 


18 


18 


Maine ^ 


37 


35 


Maryland 


28 


27 


Massachusetts 


42 


39 


Michigan ^ 


37 


35 


Minnesota ^ 


42 


41 


Mississippi 


15 


15 


Missouri 


36 


33 


Montana ^ 


46 


44 


Nebraska 


36 


38 


Nevada 


23 


22 


New Mexico 


20 


20 


New York ^ 


30 


28 


North Carolina 


27 


25 


North Dakota 


40 


38 


Ohio 


41 


39 


Oklahoma 


26 


25 


Oregon t 


33 


34 


Rhode Island 


29 


27 


South Carolina 


20 


20 


Tennessee 


25 


24 


Texas 


23 


23 


Utah 


34 


34 


Vermont ^ 


40 


39 


Virginia 


31 


29 


West Virginia 


26 


24 


Wyoming 


36 


34 


Other Jurisdictions 


American Samoa 


2 


2 


DDESS 


35 


33 


DoDDS 


37 


38 


Guam 


6 


6 



t Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: National results are based on the national sample, not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Figure 4.5 Cross-State Proficientleye\ Comparisons for Accommodations-Permitted Results, Grade 4 



Comparisons of percentage of students at or above Proficient in science for grade 4 public schools; 
2000 sample where accommodations were permitted 



Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction’s abbreviation to the key below to determine whether the percentage of students at or above Proficient m this jurisdiction is higher 
than, the same as, or lower than the jurisdiction in the column heading. For example, in the column under Virginia, the percentage of students at or above 
Proficient \r\ Virginia was lower than Massachusetts, Vermont, and Maine, about the same as all the states from Montana through Oregon, and higher than 
the remaining states down the column. 
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Jurisdiction has statistically significantly higher percentage 
than the jurisdiction listed at the top of the figure. 

No statistically significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statistically significantly lower percentage 
than the jurisdiction listed at the top of the figure. 




The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction Is being compared 
with every other jurisdiction. Significance is determined by an 
application of a multiple comparison procedure (see appendix A). 

-f-»- Indicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS; Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE; Differences between states and jurisdictions may be partially 
explained by other factors not included in this figure. 

SOURCE; National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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Figure 4.6 Cross-State Proficient Ley e\ Comparisons for Accommodations-Permitted Results, Grade 8 



Comparisons of percentage of students at or above Projident in science for grade 8 public schools: 
2000 sample where accommodations were permitted 

Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the figure. Match the shading intensity 
surrounding a jurisdiction's abbreviation to the key below to determine whether the percentage of students at or above Proficient in this jurisdiction 
is higher than, the same as, or lower than the jurisdiction in the column heading. For example, in the column under Maine, the percentage of 
students at or above Proficieit in Maine was lower than Montana and Minnesota, about the same as all the states from Ohio through Department of 
Defense Domestic Dependent Elementary and Secondary Schools (DD), and higher than the remaining states down the column. 
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c: 

□ 

□ 



Jurisdiction has statisticaiiy significantly higher percentage 
than the jurisdiction listed at the top of the figure. 

No statisticaiiy significant difference detected from the jurisdiction 
listed at the top of the figure. 

Jurisdiction has statisticaiiy significantiy lower percentage 
than the jurisdiction listed at the top of the figure. 
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The between jurisdiction comparisons take into account sampling and 
measurement error and that each jurisdiction is being compared 
with every other jurisdiction. Significance Is determined by an 
application of a multiple comparison procedure (see appendix A). 

++lndicates that the jurisdiction did not satisfy one or more of 
the guidelines for school participation rates (see appendix A). 

DDESS; Department of Defense Domestic Dependent 
Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Differences between states and jurisdictions may be partially 
explained by other factors not Included In this figure. 

SOURCE: National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP), 2000 Science Assessment. 
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Teaching and Learning Science 



During the past 15 to 20 years science education has 
undergone a number of reforms that were spurred on 
initially by the 1983 report entitled A Nation at Risk. This 
report raised the concern that national student achievement 
across core subjects was eroding.' Publications by 
organizations such as the American Association for the 
Advancement of Science (AAAS), the National Research 
Council of the Academy of Sciences (NRC), and the 

National Science Teachers Association helped focus 
attention on a number of critical issues in science 
education that ranged from what science content to 
teach to how learning should be assessed.^ These 
pubhcations, especially Benchmarks for Science Literacy 
and The National Science Education Standards, have 
been extensively used by some states as they have 
revised or created new science standards.^ Some 
recent pubhcations written by the AAAS and NRC 
build on the information contained in Benchmarks 
and the National Standards, addressing in more detail 
such topics as curriculum design, how learning 
should be assessed, and how inquiry-based learning 




What teacher 
factors are 
related to 
science achieve- 
ment? 

How does 
technology use 
and student 
course work relate 
to achievement? 

V 



^ National Commission on Excellence in Education. (1983). A nation at risk: The 
imperative for educational reform, Washington DC: Author. 

^ American Association for the Advancement of Science. (1993). Benchmarks for science 
Washington, DC: Author. 

National Research Council of the Academy of Sciences. (1995). National science 
education standards. W^ishington, DC: Author. 

National Science Teachers Association. (1992). The content core: Scope, sequence, and 
coordination ^wirfe. Washington, DC: Author 

^ For examples 

http://www.dpi.state.wi.us/standards 

http://www.isbe.state.il.us/ils/science 

http://www.state.nj.us/njded/cccs 
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helps students learn science content."^ 
However, the fact that a wealth of informa- 
tion on science teaching and learning is 
available to teachers does not necessarily 
mean that teachers incorporate such 
information into their daily classroom 
activities. Furthermore, there is a lack of 
information concerning the efficacy of 
certain teaching and learning strategies as 
they relate to what students know and can 
do in science. Thus, the results of the 
NAEP science assessment are very impor- 
tant since they give valuable information 
about teacher practices in the classroom, 
and may help to elucidate the relationship 
between those practices and student 
achievement. 

This chapter considers school factors 
related to teaching and learning, as 
reported by teachers and students and 
examines their relationship to students’ 
average scale scores on the NAEP 2000 
science assessment. The information is 
based on responses to questionnaires 
answered both by teachers of students who 
participated in the assessment and by the 
students who took the assessment. Data 
based on teachers’ responses are presented 
for grades 4 and 8 only. Grade 12 teachers 
were not administered a questionnaire 
because it is difficult to Hnk students to 
teachers across the many different science 
courses taught at this grade level. 

The information presented in this 
chapter may help readers interpret some of 
the findings found in earlier chapters of 



this report. The contexts for teaching and 
learning explore two areas: computer 
availability and use, and students’ course- 
taking practices. As with all NAEP data, 
the unit of analysis in this chapter is the 
student. Although some of the data re- 
ported here are based on teachers’ responses 
to the questionnaires, the results are re- 
ported in terms of percentages of students 
whose teachers responded to each question 
in a particular manner. The results for each 
of the factors described in this chapter 
include the percentages of students and 
their corresponding average scale scores. 
Results from the 2000 assessment are 
compared to 1996 for those questions that 
were asked of students or teachers in both 
assessment years. In some cases, data are 
available only from the 2000 assessment. 

Readers of this report are reminded that 
the relationship between a contextual 
variable and science performance is not 
necessarily causal, and that different inter- 
pretations may apply to a given finding of 
association between a variable and average 
science scores. For example, one finding 
reported in this chapter is that twelfth- 
graders who used computers to collect data 
at least once a month outperformed their 
peers who did so less fiequently. One 
possible interpretation of this finding is that 
the experience of using a computer in this 
manner for science learning may help 
students achieve in science. Conversely, it 
may also be possible that teachers of 
students who are already high achievers 



^ National Research Council of the Academy of Sciences. (2001). Classroom assessment and the national science 
education s^fjNdijrds.Washington, DC: Author. 

National Research Council of the Academy of Sciences. (2000). Inquiry and the national science education standards: A 
guide to teaching and Washington, DC: Author. 

American Association for the Advancement of Science. (2001). Designs for science //reraq/. Washington, DC: Author. 
American Association for the Advancement of Science. (2001). Atlas of science Washington, DC: Author. 
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may be more likely to allow their students 
to spend time collecting data with comput- 
ers than are teachers of lower-achieving 
students. Without further study, the exact 
cause for the relationship between this 
instructional practice and students’ average 
science scores cannot be determined. 

Technology Use: 

Availability of Computers 
for Science Classes 

How to best use computers for teaching 
and learning is an ongoing discussion 
among educators. There are many issues 
associated with effective use of technology 
in the classroom that range from computer 
access to teachers’ expertise in building the 
tools computers offer into their teaching 
and learning plans. ^ This multitude of 
factors makes it difficult to assess the 
effectiveness of computers vis-a-vis student 
learning. While the data on computers that 
were collected as part of the NAEP 2000 
science assessment do not pretend to 
answer the questions these issues raise, the 
data do give an indication of how teachers 
are using computers. The following section 
reports some of these findings. Other data 
about computer availability and use can 
be found on the NAEP web site at 
http:/ / nces.ed.gov/nationsreportcard. 



Teachers of students in grades 4 and 8 
were asked which best described the 
availabihty of computers for use by their 
science students. The response options are 
shown in table 5.1, together with the 
percentage of students whose teachers 
chose each response option and students’ 
average science scores. In 2000, only 11 
percent of fourth-graders and 10 percent 
of eighth-graders were taught by teachers 
who reported that no computers were 
available for use by the science students. 
Approximately 24 percent of fourth- 
graders and 41 percent of eighth-graders 
were taught by teachers who indicated that, 
although computers were available in a 
laboratory, they may not have had comput- 
ers in their classrooms. Between 1996 and 
2000, none of the apparent changes in the 
availability of computers at grades 4 and 8 
were found to be statistically significant. 

At both grades 4 and 8 in 2000, students 
who could access computers in laboratories 
scored higher, on average, than their peers 
who had no access at aU to computers. 
Regardless of the number of computers 
teachers reported having in their class- 
rooms, there was no statistically significant 
difference detected in the average scores of 
students who had no access to computers 
and those who had one or more computers 
available in their classrooms. 



^ National Science Teachers Association. (1992). NSTA position statement: The use of computers in science education 
[Online]. Available; http://www.nsta.Org/l 5 9&id= 4 

Mayer, D.P., Mullens, J.E., Moore, M.T., & Mathematics Policy Research, Inc. (2000). Monitoring school quality: An 
indicators report Washington, DC: U.S. Department of Education, Office of Educational Research and Improve- 
ment, National Center for Education Statistics. 
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Table 5.1 

Percentage of fourth- and 
eighth-graders and average 
scale score by teachers’ reports on 
availability of computers for use by 
their science students: 1996 and 2000 




Available in a computer laboratory and 
easy to access or schedule 

The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Technology Use: 

Computers for Instruction in 
Science, Grades 4 and 8 

Teachers whose students participated in the 
science assessment were asked how they 
used the computer for instruction in 
science. Since they could identify more 
than one type of computer use, the results 
are reported in terms of a ‘‘yes” or “no 
response” for each type of computer use. 
Table 5.2 shows the percentages and 
average scores of students whose teachers 
reported using the computer for drill and 
practice, playing science/learning games, 
simulations and modeling, data analysis and 
other applications, and word processing. It 
also provides the data for students whose 
teachers stated that they did not use com- 
puters for science instruction. It is impor- 
tant to note that any apparent relationship 
between computer use and student perfor- 
mance may reflect the influence of factors 
other than the type of computer use in and 
of itself. 

In 2000, 43 percent of fourth-graders 
and 26 percent of eighth-graders had 
teachers who did not use computers for 
science instruction. At grade 4, students 
whose teachers indicated that they did not 
use computers for science instruction 
scored lower, on average, than did students 
whose teachers did use computers. (Note 
that a “no response” to this option in the 
table indicates that computers were used by 
teachers for science instruction.) 



In 2000, fourth-graders whose teachers 
indicated that they used computers for 
playing science/learning games scored 
higher, on average, than fourth-graders 
whose teachers indicated that they did not 
use computers in this manner during 
science instruction. Eighth-graders whose 
teachers indicated using computers for 
simulations and modeling, and data analysis 
and other applications scored higher, on 
average, than eighth-graders whose teach- 
ers did not indicate doing so. 

The results presented in table 5.2 also 
indicate an overall increase between 1996 
and 2000 in the percentage of both fourth- 
and eighth-graders whose teachers re- 
ported using computers for science instruc- 
tion. The percentage of students whose 
teachers indicated using computers in- 
creased from 47 to 57 percent at grade 4, 
and from 54 to 74 percent at grade 8. Also 
at grade 8, there was an increase in the 
percentage of students whose teachers said 
they used computers for data analysis and 
other applications, and for word processing, 
as a part of science instruction. 
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Table 5.2 

Percentage of fourth- and 
eighth-graders and average scale score 
by teachers’ reports on how they use 
computers for science instruction: 
1996 and 2000 





Fourth-graders 
whose teachers used 
computers for playing 
science/learning 
games had higher 
average scores than 
fourtitgraders 
whose teachers did 
not use computers in 
this manner. 

Eighth-graders 
whose teachers used 
computers for 
simulations and 
modeling, and for 
data analysis and 
other applications, 
as a part of science 
instruction had 
higher average 
scores than eighth- 
graders whose 
teachers did noliise 
computers in this 
manner. 



The percentage of students is listed first with the corresponding average scale score presented below. 

* Significantly different from 2000. Although not marked in the table, the difference in the percentage of students not responding in 1996 is significantly 
different from 2000 in all instances where the corresponding percentage of students responding yes is significantly different. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Technology Use: 

Computers for Instruction in 
Science, Grade 12 

In 2000, twelfth-grade students were asked 
how frequently they used computers in 
their science classes for collecting data 
using lab equipment that interfaces with 
computers; downloading data and related 
information from the Internet; analyzing 
data using the computer; and using the 
Internet to exchange information with 
other students or scientists about science 
experiments or investigations. The results 
are shown in table 5.3. 

Thirty-four percent of twelfth-graders 
reported that they were not taking a 
science course, and between 42 and 54 
percent of students stated that they never 
used computers to do the listed activities. 
The remaining percentage of students were 
fairly evenly split between those who 
indicated that they used the computer for 
each of the Hsted activities at least once a 
month and those who did so less than once 
a month. 



With one exception, students who 
reported that they were not taking a 
science course were outperformed by their 
peers who were, even when their peers 
were not using computers for the listed 
activities. Average scores for students who 
reported using the Internet to exchange 
information with other students once a 
month or more were not found to be 
significantly diflferent from those of stu- 
dents who were not taking science. 
Twelfth-graders who reported collecting 
data and who reported analyzing data with 
computers at least once a month outper- 
formed their peers who reported doing so 
less frequently. Students who said they 
never downloaded data and related infor- 
mation from the Internet scored lower, on 
average, than their peers who indicated 
doing so at least sometimes. 
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Table 5.3 

Percentage of twelfth-graders 
and average scale score by 
students reports on how they 
use computers in science classes: 
2000 



Grade 



Computer Use 



Collect data using lab equipment that interfaces with computers 




1 am not taking science 


34 

141 


Once a month or more 




Sometimes but less than once a month 




Never 


42 \ 

148 


Download data and related information from the Internet 




1 am not taking science 


34 

142 


Once a month or more 


9 

155 


Sometimes but less than once a month 


13 

158 


Never 


45 

148 


Analyze data using the computer 




1 am not taking science 


/ 


Once a month or more 




Sometimes but less than once a month 


T\ 

157 


Never 


44 

147 


Use the Internet to exchange information with other students 
or scientists about science experiments or investigations 




1 am not taking science 


34 

142 


Once a month or more 


4 

146 


Sometimes but less than once a month 


7 

151 


Never 


54 




151 



Twelfth-graders who 
said they used 
computers to collect 
data or to analyze 
data at least once 
a month had higher 
average scores 
than twelfth-graders 
who did so less 
frequently. 



The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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student Coursework: 

Grade 4 Science Courses 

Students in grade 4 tend to take a science 
course that incorporates a mixture of topics 
in science. While increasing emphasis on 
state standards and state assessments may 
have led to the topics covered in the early 
grades becoming more formalized, the 
topics taught in fourth-grade science 
classes are not necessarily unified across the 
nation.^ Since the instructional information 
that can be collected from teachers on a 
questionnaire is somewhat limited, infor- 
mation collected by NAEP on science 
course work for fourth-graders was con- 
fined to asking teachers how much time 
was spent on the broad domains of Life 
science. Earth science, and physical science. 
Teachers responding to this question could 
choose from the options “a lot,” “some,” 
“little,” and “none.” It is important to note 
that the responses did not refer to minutes 
or hours spent on the domain, but rather to 
time spent in relation to the other areas. 
Thus, teachers may have spent only 10 
minutes a week on life science and stiU 
have indicated that they devoted “a lot” of 
time to this domain. The results for this 
question are presented in table 5.4. 



In 2000, teachers of 31 percent of 
fourth-graders reported spending a lot of 
time on life science and Earth science, and 
teachers of 22 percent of fourth-graders 
reported spending a lot of time on physical 
science. A very small percentage of fourth- 
graders were taught by teachers who said 
they actually devoted no time to any of 
these three science domains — only 1 to 2 
percent. The amount of time teachers 
devoted to life science and to Earth science 
displayed a fairly positive relationship with 
average NAEP science scores. In both cases, 
students whose teachers indicated that they 
devoted a lot or some time on these 
science domains outperformed their peers 
whose teachers indicated spending little 
time. 

The percentage of students whose 
teachers reported spending a lot of time on 
Earth science increased from 19 percent in 
1996 to 31 percent in 2000. At the same 
time, the percentage of students whose 
teachers reported spending only some time 
on the Earth science domain decreased 
from 76 to 62 percent. 



^ Council of Chief State School Officers, Wisconsin Center for Education Research, Eleven State Collaborative. 
(2000). Using data on enacted curriculum in mathematics & science: Sample results from a study of classroom practices and 
subject content. Summary report from Survey of Enacted Curriculum Project. Washington, DC: Council of Chief State 
School Officers. 

National Committee on Science Education Standards and Assessment, National Research Council. (1996). 
National science education standards. Washington, DC: National Academy Press. 

O’Sullivan, C.Y., Weiss, A. R., Askew, J.M. (1998). Students learning science. Washington, DC: U.S. Department of 
Education. Office of Educational Research and Improvement. National Center for Education Statistics. 
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Table 5.4 

Percentage of fourth-graders and average 
scale score by teachers’ reports on how 
much time is spent on certain 
science domains: 1996 and 2000 



Grade 



Time Spent 
Teaching Certain 
Science Domains 




Fourth-graders 
whose teachers 
spent at least some 
time on lif e science 
and earth science 
had higher average 
scores than fourth- 
graders whose 
teachers spent only 
a little time on these 
science domains. 



The percentage of students is listed first with the corresponding average scale score presented below. 

* Significantly different from 2000. 

— Sample size is insufficient to permit a reliable estimate, 

A Percentage is between 0.0 and 0.5. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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student Coursework: 

Grade 8 Science Courses 

By the time students reach middle school, 
science is being taught as a core content 
area. There is, however, no consensus as to 
the order in which the key domains should 
be taught.*^ As part of the NAEP science 
assessment, eighth-grade teachers were 
asked the same question that was asked of 
teachers of fourth-graders, namely to 
indicate how much time they spent on 
certain science domains. Readers are 
reminded that the NAEP assessment 
surveys the content domains of Earth, 
physical, and life science; thus, if students 
are to do well on NAEP, breadth of cover- 
age may be important in middle school. 

Table 5.5 presents the percentages of 
eighth-graders and their average scores by 
teachers’ reports on how much time they 
spent on various science domains. In 2000, 
45 and 47 percent of eighth-graders were 



taught by teachers who spent a lot of time 
on Earth science and physical science, 
respectively. Twenty-one percent of 
eighth-graders were taught by teachers 
who indicated spending a lot of time on 
hfe science. None of the apparent changes 
between 1996 and 2000 in eighth-grade 
teachers’ reports of amount of time devoted 
to any of the science domains were statisti- 
cally significant. 

In 2000, the relationship between 
teachers’ reports on the amount of time 
devoted to the various science domains and 
eighth-graders’ average science scores was 
somewhat different than that observed at 
the fourth grade. For both hfe science and 
Earth science, the students whose teachers 
reported spending no time on these 
domains outperformed their peers 
whose teachers reported spending a lot 
or some time. 



’ Council of Chief State School Officers, Wisconsin Center for Education Research, Eleven State Collaborative. 
(2000). Using data on enacted curriculum in mathematics & science: Sample results from a study of classroom practices and 
subject content. Summary report from Survey of Enacted Curriculum Project. Washington, DC: Council of Chief State 
School Officers. 

National Committee on Science Education Standards and Assessment, National Research Council. (1996). 
National science education srandards. Washington, DC: National Academy Press. 

O’Sullivan, C. Y., Weiss, A. R., & Askew, J.M. (1998). Students learning Washington, DC: US. Department of 

Education, Office of Educational Research and Improvement, National Center for Education Statistics. 
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Table 5.5 

Percentage of eighth-graders and average 
scale score by teachers’ reports on how 
much time is spent on certain 
science domains: 1996 and 2000 



Grade 




Time Spent 
Teaching Certain 
Science Domains 




Eighth-graders 
whose teachers said 
they spent no time 
on life science or 
Earth science had 
higher average 
scores than eighth- 
graders whose 
teachers spent at 
4eas^some time on 
these science 
domains. 
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student Coursework: 

Grade 12 Science Courses 

Most states have science coursev^ork 
requirements for graduation; however, in 
some states the requirements are deter- 
mined at the local level. In 2000, according 
to a Council of Chief State School Officers 
report, 4 states required four credits for 
graduation, 15 states and the Department 
of Defense schools required three credits, 

21 required two credits, and 2 required 
only one credit.® The number of science 
credits required in the remaining states 
were either determined by a local board or 
were included as part of combined credits 
in mathematics and science. Some states 
required that students take specific courses 
such as life science and physical science, 
while other states made no such demands. 
Some states required that students take 
courses in specific areas such as life science 
and physical science. However, course 
requirements in life science and physical 
science can often be fulfilled without 
taking a core course in biology, chemistry, 
or physics. While seven states did require at 
least one of the core science courses, no 
state required aU three for graduation.^ 



Twelfth-grade students in the NAEP 
science assessment responded to several 
questions about their science studies. They 
were asked whether they were currently 
taking a science class and then asked to 
indicate which courses they had taken from 
the eighth grade to the present. The list of 
courses included Earth and space science, 
life science (other than biology), physical 
science (other than physics and chemistry), 
general science, integrated science, biology, 
chemistry, physics, and science and technol- 
ogy. Students were also asked if they were 
currently enrolled in or had taken Ad- 
vanced Placement courses in biology, 
chemistry, and physics. The data collected 
from these questions are presented on the 
following pages. 



® Council of Chief State School Officers. (2000). Key state education policies on K~i2 education: 2000:T\\ne and 
attendance, graduation, content standards, teacher & school licensure, student assessment. Washington, DC: Author. 

^ Ibid. 
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Table 5.6 



Percentage of twelfth-graders and average 
scale score by students’ reports on whether 
or not taking a science course this year: 
1996 and 2000 




Science Course 
Taking 



1996 2000 




Twelfth-graders who 
were enrolled in a 
science course had 
higher average 
scores than twelfth- 
graders who were 
not. 



The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Current Science Course Enrollment 

Table 5.6 shows the percentages of students 
who were enrolled in a science course in 
2000 and in 1996. As can be seen from the 
data, in 2000, 53 percent of students re- 
ported currently taking a science course, 
whereas 47 percent reported not taking 
one. This was similar to the results in 1996. 
Students who reported that they were 
currently taking a science course in 2000 
outperformed their counterparts 
who reported that they were not taking 
a science course at the time of the 
assessment. 



Science Courses Taken Since 
Eighth Grade 

Table 5.7 presents the results for a question 
that asked students in which grade they 
had taken certain science courses. The 
grades covered were 8 through 12. Students 
were also asked to indicate if they had not 
taken a specific course. The actual list 
presented to students included more courses 
than are listed in the table; for example 
second year biology was included on the 
list that was presented to students, but is 
not presented in table 5.7. A complete 
listing can be found on the NAEP web site. 
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Science Course 
Taken 



Table 5.7 

Percentage of twelfth-graders and average 

scale score by students’ reports on science Grade 

courses taken since eighth grade: 2000 





Not taken 


Grade 8 


Grade 9 


Grade 10 


Grade 11 


Grade 12 


Earth (and space) science 


26 


49 


19 


5 


4 


3 




148 


150 


146 


135 


140 


144 


First-year biology 


8 


2 


31 


54 


5 


1 




126 


138 


156 


149 


134 


125 


First-year chemistry 


30 


1 


2 


21 


40 


7 




128 


128 


144 


166 


155 


145 


First-year physics 


64 


1 


2 


2 


12 


19 




139 


128 


153 


159 


167 


167 


Life science (other 


46 


22 


18 


10 


6 


5 


than biology) 


151 


152 


139 


131 


141 


157 


Physical science (other 


36 


12 


36 


11 


6 


3 


than chemistry and physics) 


151 


159 


147 


135 


132 


141 


General science 


47 


37 


14 


4 


2 


1 




148 


152 


145 


129 


134 


144 


Integrated science 


85 


5 


7 


3 


1 


1 




149 


147 


149 


132 


135 


142 


Science and technology 


86 


4 


4 


3 


4 


4 




148 


154 


154 


147 


148 


149 



The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE; Row percentages may not add to 100 because some students indicated taking a course in more than one grade. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 




The table shows that 26 percent of 
students reported not taking Earth and 
space science in grades 8 through 12. This 
does not mean that they never had a course 
in earth and space science. They may have 
taken it in grade 7, or even in grade 6. 
Almost one-half of the twelfth-grade 
student population did report taking the 
course during eighth grade. While almost 



all students had taken biology at some 
point since the eighth grade, the most 
popular grade for taking the course was 
tenth. Two- thirds of twelfth-graders 
reported taking chemistry. Forty percent of 
students took the course in the eleventh 
grade. Thirty-six percent of twelfth- 
graders reported taking physics — most 
typically in the eleventh or twelfth grade. 
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Enrollment in Advanced Placement 
Science Courses 

Many schools offer higher-level courses 
that allow students to accumulate college 
credits. Table 5.8 displays the percentage of 
students in 2000 who reported that they 
were currently enrolled in or had taken an 
Advanced Placement course in the three 
core sciences — biology, chemistry, and 
physics. Students’ average scores are also 
presented. 



The results show that 10 percent of 
twelfth-graders had taken or were enrolled 
in biology, and that 6 and 5 percent had 
taken or were enrolled in chemistry and 
physics, respectively. Students who had 
taken or were enrolled in AP biology, 
chemistry, or physics scored higher, on 
average, than those students who said they 
had not taken and were not enrolled in 
these courses. 



Table 5.8 








Percentage of twelfth-graders and average 
scale score by students’ reports on whether 
they are currently enrolled in or have taken 
an Advanced Placement course: 2000 


Grade ^ 


1 


Students’ Reports 
on Advanced 
Placement Courses 





Yes No response 




Twelfth-graders who 
had taken an AP 
course in biology, 
chemistry, or 
physics had higher 
average scores than 
twelfth-graders who 
had not taken one of 
these courses. 



The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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A Appendix A 

The NAEP 2000 Science Assessment 



The design of the NAEP 2000 science assessment followed 
the guidehnes provided in the framework developed for the 
1996 assessment. While maintaining some conceptual 
continuity with the NAEP 1990 science assessment, the 
1996 framework took into account the current reforms in 
science education, as well as documents such as the science 
framework used for the 1991 International Assessment of 
Educational Progress. In addition, the Framework 
Steering Committee recommended that a variety of 
strategies be used for assessing students’ performance. 
These included: 

■ performance tasks that allow students to 
manipulate physical objects and draw scientific 
understanding from the materials before them; 

■ constructed-response questions that provide 
insight into students’ levels of understanding and 
ability to communicate in the sciences as well as 
their ability to generate, rather than simply 
recognize, information related to scientific 
concepts and their interconnections; and 

■ multiple-choice questions that probe students’ 
conceptual understanding and ability to connect 
ideas in a scientifically sound way. 

Samples of each type of task and question are available in 
the “NAEP Questions” section of the NAEP web site 
http://nces.ed.gov/nationsreportcard.The framework for 
the 1996 and 2000 science assessments is represented as a 
matrix with two dimensions: 1) fields of science (Earth, 
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physical, and life) and 2) elements of 
knowing and doing science (conceptual 
understanding, scientific investigation, and 
practical reasoning). The fields-of-science 
dimension is used to create three subscales 
at each grade. Subscales are not created 
based on the elements of knowing and 
doing science. In addition there are two 
overarching domains that describe science: 
1) nature of science and 2) themes. [These 
overarching domains provide additional 



guidance to the development of assessment 
questions and tasks, ensuring that the 
assessment also integrates the three fields of 
science rather than only represents three 
separate content areas.] Figures A. la, A. lb, 
and A.lc describe, respectively, the fields of 
science, the elements of knowing and 
doing science, and the overarching domains 
that guided the development of the 1996 
and 2000 science assessments. 
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Figure A.ia 



Descriptions of the Three Fields of Science 



Earth Science 



The content of Earth science falls under the general headings of the solid Earth 
(lithosphere), water (hydrosphere), air (atmosphere), and the Earth in space. Topics 
related to the solid Earth include the composition of the earth; forces that alter its 
surface; the formation, characteristics and uses of rocks; the changes and uses of 
soil; natural resources used by humankind; and natural forces within the Earth. 
Topics related to water include the water cycle; the nature of oceans and their 
effects on water and climate; and the location of water, its distribution, character- 
istics, and effect of and influence on human activity. Topics related to air include 
the composition and structure of the atmosphere (including energy transfer); the 
nature of weather; common weather hazards; and air quality and climate. Topics 
related to the Earth in space include the setting of the Earth in the solar system; 
the setting and evolution of the solar system in the universe; tools and technology 
that are used to gather information about space; the apparent daily motions of the 
Sun, the Moon, the planets and the stars; the rotation of the Earth about its axis, 
and the Earth's revolution around the Sun; and the tilt of the Earth's axis that 
produces seasonal variations in the climate. 



Physical Science 



The physical science component covers basic knowledge and understanding 
concerning the structure of the universe as well as the physical principles that 
operate within it. The major topics are matter and its transformations, energy and 
its transformations, and the motion of things. Matter and its transformations are 
described by diversity of materials (classification and types and the particulate 
nature of matter); temperature and states of matter; properties and uses of material 
(modifying properties, synthesis of materials with new properties); and resource 
management. Energy and its transformations includes different forms of energy; 
energy transformations in living systems, natural physical systems, and artificial 
systems constructed by humans; and energy sources and use, including distribu- 
tion, energy conversion, and energy costs and depletion. Motion includes frames of 
reference; force and changes in position and motion; action and reaction; vibra- 
tions and waves as motion; general wave behavior; electromagnetic radiation; and 
the interactions of electromagnetic radiation with matter. 



Life Science 



L 



The fundamental goal of life science is to understand and explain the nature and 
function of living things. The major concepts assessed in life science are change 
and evolution, cells and their functions (not at grade 4), organisms, and ecology. 
Change and evolution includes diversity of life on Earth; genetic variation within a 
species; theories of adaptation and natural selection; and changes in diversity over 
time. Cells and their functions covers information transfer; energy transfer for the 
construction of proteins; and communication among cells. Organisms covers 
reproduction, growth and development; life cycles; and functions and interactions 
of systems within organisms. Ecology focuses on the interdependence of life — 
populations, communities, and ecosystems. 



SOURCE: National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of Educational Progress. 
Washington, DC: Author. 
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Figure A.ib 



Descriptions of Knowing and Doing Science 



Conceptual 

Understanding 


Conceptual understanding includes the body of scientific knowledge that students 
draw upon when conducting a scientific investigation or engaging in practical 
reasoning. Essential scientific concepts involve a variety of information including 
facts and events the student learns from science instruction and experiences with 
the natural environment and scientific concepts, principles, laws, and theories that 
scientists use to explain and predict observations of the natural world. 


Scientific 

Investigation 


Scientific investigation probes students' abilities to use the tools of science, 
including both cognitive and laboratory tools. Students should be able to acquire 
new information, plan appropriate investigations, use a variety of scientific tools, 
and communicate the results of their investigations. 


Practical 
^ Reasoning 


Practical reasoning assesses students' ability to use and apply science understand- 
ing in new, real-world applications. 


SOURCE: National Assessment Governing Board. (2000). Science Framework for the 1996 and 2000 National Assessment of Educational Progress. 
Washington, DC: Author. 


Figure A.ic 


Description of Dverarching Domains 


The Nature of 
Science 


The nature of science incorporates the historical development of science and 
technology, the habits of mind that characterize these fields, and methods of 
inquiry and problem-solving. It also encompasses the nature of technology and 
includes issues of design, application of science to real-world problems, and trade- 
offs or compromises that need to be made. 


Themes 


Themes are the "big ideas" of science that transcend the various scientific 
disciplines and enable students to consider problems with global implications. The 
NAEP science assessment focuses on three themes: systems, models, and patterns 
of change. 




• Systems are complete, predictable cycles, structures, or processes occurring in 
natural phenomena. Students should understand that a system is an artificial 
construction created to represent or explain a natural occurrence. Students should 
be able to identify and define the system boundaries, identify the components 
and their interrelationships, and note the inputs and outputs to the system. 




• Models of objects and events in nature are ways to understand complex or 
abstract phenomena. As such they have limits and involve simplifying assump- 
tions but also possess generalizability and often predictive power. Students need 
to be able to distinguish the idealized model from the phenomenon itself and to 
understand the limitations and simplified assumptions that underlie scientific 
models. 


L ^ 


• Patterns of change require students to recognize patterns of similarity and 
differences and to recognize how these patterns change over time. In addition, 
students should be able to remember common types of patterns and transfer 
their understanding of a familiar pattern of change to a new and unfamiliar one. 
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Table A. la summarizes the distribution of 
assessment time across the three fields of 
science — ^Earth, physical, and life. These 
fields provide the basis for the content area 
scales. Care was taken to ensure congru- 
ence between the percentages used in the 



assessment (actual) and those indicated in 
the assessment specifications (target). The 
classification of items by field of science 
was overseen and approved by a committee 
of expert science educators. 



Table A.ia 



Distribution of assessment time by field of science: 1996 and 2000 







Earth 






Physical 






Life 






Target 


Actual 

1996 


Actual 

2000 


Target 


Actual 

1996 


Actual 

2000 


Target 


Actual 

1996 


Actual 

2000 


Grade 4 


33% 


33% 


33% 


33% 


34% 


33% 


33% 


33% 


33% 


Grade 8 


30% 


30% 


31% 


30% 


30% 


34% 


40% 


40% 


35% 


Grade 12 


33% 


33% 


33% 


33% 


33% 


31% 


33% 


34% 


37% 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Table A. lb shows the distribution of 
assessment time across the second dimen- 
sion: knowing and doing science. This 
dimension includes conceptual understand- 
ing, scientific investigation, and practical 
reasoning. As with the above classification 
of items, an expert committee of science 
educators oversaw the categorization of 
items by this dimension. In both this table 



and the table above, variation is evident 
across the two assessment years in percent- 
ages of questions within categories. Such 
variation is the result of releasing several 
blocks of questions from the 1996 assess- 
ment and replacing them with newly 
developed questions in 2000. In addition, 
one of the four hands-on blocks adminis- 
tered at each grade in 1996 was released. 



Table A.ib 



Distribution of assessment time by knowing and doing science: 1996 and 2000 



Conceptual understanding Scientific investigation Practical reasoning 





Target 


Actual 

1996 


Actual 

2000 


Target 


Actual 

1996 


Actual 

2000 


Target 


Actual 

1996 


Actual 

2000 


Grade 4 


45% 


45% 


56% 


45% 


38% 


27% 


10% 


17% 


17% 


Grade 8 


45% 


45% 


59% 


30% 


29% 


18% 


25% 


26% 


24% 


Grade 12 


45% 


44% 


56% 


30% 


28% 


24% 


25% 


28% 


20% 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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and no replacement block was developed 
for 2000. This resulted in a smaller propor- 
tion of scientific investigation questions at 
each grade in 2000 than in 1996. The 
reporting of changes in student perfor- 
mance is not affected by these variations 
because trend reporting is based upon the 
underlying scale, which uses the common 
blocks (i.e., those used in both assessment 
years), but maintains its stability even if 
some blocks are dropped or replaced. 

The Assessment Design 

One-half of the students who participated 
in the science assessment received a book- 
let containing six sections; the other half, 
five sections. All the booklets contained 
either two or three sections that were 
blocks of cognitive questions assessing 
knowledge and skills outlined in the 
framework. In addition, each booklet 
contained two sections that were sets of 
background questions. Each booklet had 
two cognitive sections containing only 
paper-and-pencil questions. The booklets 
with three blocks of cognitive questions 
also contained a hands-on task with related 
paper-and-pencil questions. The booklets 
with two blocks of cognitive questions did 
not contain a hands-on task. Thus, one-half 
of the students who participated in the 
assessment performed a hands-on task. 

At each grade level there were 14 
different sections or blocks of cognitive 
questions usually consisting of both mul- 
tiple-choice and constructed-response 
questions.^ Short constructed-response 
questions required a few words or a sen- 
tence or two for an answer (e.g., briefly 



stating why a potted plant can survive in a 
sealed container much longer than a 
mouse), while extended constructed- 
response questions generally required a 
paragraph or more (e.g., outlining an 
experiment to find the density of a metal 
ring). Some extended constructed-response 
questions also required diagrams, graphs, or 
calculations. It was expected that students 
could adequately answer the short con- 
structed-response questions in about two 
to three minutes and the extended con- 
structed-response questions in about five 
minutes. 

Other features were built into the blocks 
of questions. Three of the blocks at each 
grade level were hands-on tasks where 
students were given a set of equipment and 
asked to conduct an investigation and 
answer questions relating to the investiga- 
tion. One-half of the students conducted a 
hands-on task that was always presented as 
the third cognitive section. A second 
feature was the inclusion of theme blocks 
at each grade level — one assessing systems, 
one assessing models, and one assessing 
patterns of change. A theme block contains 
a set of questions that all focus on a par- 
ticular theme, and requires students to 
engage more thoroughly in the topics 
related to that theme. For example, stu- 
dents were asked to make drawings and 
graphs based on data given about the solar 
system and then answer a number of 
questions. Theme blocks were placed 
randomly in the student booklets, but did 
not appear in every booklet. No student 
received more than one theme block. 



* These 14 blocks were distributed across the student booklets in a Balanced Incomplete Block (BIB) design that is 
described later in this section. 
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The data in table A. 2 display the number 
of questions by type and by grade level for 
the 1996 and 2000 assessments. Some of 
these questions were used at more than one 
grade level; thus, the sum of the questions 
that appear at each grade level is greater 
than the total number of unique questions. 
The total number of questions at each 
grade level in 2000 is up from 1996. This 
increase was possible because more mul- 
tiple-choice questions that take less time 
were used in 2000. This increase in mul- 
tiple-choice questions across the entire 
assessment was due to the fact that the 
blocks developed for 2000 to replace those 
released from the 1996 assessment con- 
tained a greater proportion of multiple- 
choice questions. In addition, as mentioned 
earher, one of four hands-on blocks at 
each grade in 1996 was released and not 
replaced for 2000. These hands-on blocks 
contain only constructed-response ques- 



tions. As a consequence, the total number 
of constructed-response questions in 2000 
was less than that in 1996. It should be 
noted that these variations across years do 
not affect the ability of NAEP to report 
trends in students’ performance across years. 
Trend reporting is based on those blocks 
that were common across the two years. 

The assessment design allowed for 
maximum coverage of science content at 
grades 4, 8, and 12, while minimizing the 
time burden for any one student. This was 
accomplished through the use of matrix 
sampling of questions, in which representa- 
tive samples of students took various 
portions of the entire pool of assessment 
questions. Individual students were re- 
quired to take only a small portion of the 
assessment, but the aggregate results across 
the entire assessment allowed for broad 
reporting of science abilities for the tar- 
geted population. 



Table A.2 


Distribution of questions administered by question type: 1996 and 2000 


m 


1 




Grade 8 
only 


1 


n 


Grade 12 
only 


r 


2000 


1996 2000 


1996 


2000 


1996 2000 


1996 


2000 


1996 


2000 


1996 


MC 42 62 


9 


9 












51 


71 


Grade 4 SCR^ 57 49 


16 


16 












73 


65 


ECR^ 12 3 


4 


4 












16 


7 


MC' 


9 


9 


44 65 


21 


21 






74 


95 


Grade 8 SCR^ 


16 


16 


58 49 


26 


26 






100 


91 


ECR^ 


4 


4 


13 3 


3 


3 






20 


10 


MC' 








21 


21 


49 


70 


70 


91 


Grade 12 SCR^ 








26 


26 


62 


57 


88 


83 


ECR^ 








3 


3 


27 


18 


30 


21 



‘ Multiple-choice questions. 

^ Short constructed-response questions. 

^ Extended constructed-response questions. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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In addition to matrix sampling, the 
Balanced Incomplete Block (BIB) design 
also balances the order of presentation of 
the blocks of questions, except for the 
hands-on blocks, which always appear in 
position three of a booklet. Furthermore, 
the design was set up to ensure that no 
student answered more than one theme- 
based block (though some students did not 
receive any). This design allows for some 
balancing of the impact of context and 
fatigue effects to be measured and reported, 
but makes allowance for the difficulties and 
disruption of administering hands-on 
blocks. It also takes into account the 
limited breadth of content coverage in- 
cluded in the theme blocks.^ 

Each booklet in the assessment also 
included two sections of student back- 
ground questions. The first section, consist- 
ing of general background questions, asked 
students about their race/ethnicity, 
mothers and father s level of education, 
reading materials in the home, homework, 
school attendance, and, at grade 12, aca- 
demic expectations.^ The second section 
asked students questions about their science 
classroom activities (e.g., hands-on exer- 
cises, courses taken, and use of specialized 
resources such as computers). 

In addition to the student assessment 
booklets, four other instruments provided 
data relating to the assessment: a teacher 
questionnaire, a school characteristics and 
poUcy questionnaire, a questionnaire 
designed to gather information about 
students with disabilities (SD) and/or 
limited English proficient (LEP) students, 
and a department chair/lead teacher 
questionnaire at grade 12. 



The teacher questionnaire was adminis- 
tered to the science teachers of the fourth- 
and eighth-grade students participating in 
the assessment. The questionnaire consisted 
of three sections and took approximately 
20 minutes to complete. The first section 
focused on the teachers general back- 
ground and experience; the second section, 
on the teacher s background related to 
science; and the third section, on classroom 
information about science instruction. 

The school characteristics and policy 
questionnaire was given to the principal or 
other administrator in each participating 
school and took about 20 minutes to 
complete. The questions asked about school 
policies, programs, facihties, and the demo- 
graphic composition and background of 
the student body. 

The SD and/ or LEP student question- 
naire was completed by a school staff 
member knowledgeable about those 
students who were selected to participate 
in the assessment and who were identified 
as: 1) having an Individualized Education 
Program (lEP) or equivalent program 
(for reasons other than being gifted and 
talented) or 2) being limited English 
proficient (LEP). A questionnaire was 
completed for each SD and/or LEP stu- 
dent sampled regardless of whether the 
student participated in the assessment. Each 
questionnaire took approximately three 
minutes to complete and asked about the 
student and the special programs in which 
he or she participated. 



2 For further details on the booklet design, see the forthcoming NAEP 2000 Technical Report. 

3 Some questions, such as those referring to parental education, were not asked of fourth-graders. 
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The department chair/lead teacher 
questionnaire was given to the high school 
science department chair or lead teacher in 
each participating high school. Previous to 
the 2000 assessment, NAEP had not 
attempted to collect information from 
teachers of twelfth-grade science, partly 
due to the difEculty in identifying the 
science teachers of assessed twelfth-graders. 
The questionnaire took about 20 minutes 
to complete. The questions asked about 
the certification of the teachers, science 
courses offered, use of computers in the 
classroom, teacher preparation time, and 
frequency of textbook replacement. As 
this was NAEP’s first attempt to collect 
information from department chairs or 
lead teachers, an official report of those 
data is not currently planned. The data 
are available on NAEP’s web site at 
http:// nces.ed.gov/ nationsreportcard 
through the data tool function. 

National and State Samples 

National Sample 

The national results presented in this report 
are based on a nationally representative 
probability sample of fourth-, eighth-, and 
twelfth-grade students.'^ The sample was 
chosen using a multistage design that 
involved sampling students from selected 
schools within selected geographic areas 
across the country. The sample design had 
the following stages: 

1) selection of geographic areas (a county, 
group of counties, or metropolitan 
statistical area); 



2) selection of schools (public and nonpub- 
lic) within the selected areas; and 

3) selection of students within selected 

schools. 

Each selected school that participated in 
the assessment and each student assessed 
represents a portion of the population of 
interest. Sampling weights are needed to 
make valid inferences between the student 
samples and the respective populations 
from which they were drawn. Sampling 
weights account for disproportionate 
representation due to the oversampling of 
students who attend schools with high 
concentrations of black and/or Hispanic 
students and students who attend nonpub- 
lic schools. Among other uses, sampling 
weights also account for lower sampHng 
rates for very small schools and are used to 
adjust for school and student nonresponse.^ 

A special feature of the 1996 and 2000 
national assessments of science was the 
collection of data from samples of students 
where assessment accommodations for 
special-needs students were not permitted 
and from samples of students where 
accommodations for special-needs students 
were permitted. NAEP inclusion rules 
were applied, and accommodations were 
offered only when a student had an 
Individualized Education Program (lEP) 
because of a disability and/or was 
identified as being a limited English 
proficient student (LEP);aU other students 
were asked to participate in the assessment 
under standard conditions. 



The student samples from American Samoa, Department of Defense Domestic Dependent Elementary and 
Secondary Schools, Department of Defense Dependents Schools (Overseas), Guam, and the Virgin Islands are not 
included in the national sample, 

5 Additional details regarding the design and structure of the national and state samples will be included in the 
forthcoming NAEP 2000 Technical Report. In addition, the reader may consult the NAEP 1998 Technical Report for 
a discussion of sampling procedures that are mostly common to aU NAEP assessments. 
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Table A.3 shows the number of students 
included in the national samples for the 
NAEP science assessments at each grade 
level. For the 1996 and 2000 assessments, 
the table includes the number of students 
in the sample where accommodations were 
not permitted and the number of students 
in the sample where accommodations were 
permitted. The table shows that the same 
non-SD and/ or LEP students were 



included in both samples in 2000; only the 
SD and/ or LEP students diflfered between 
the two samples. The 1996 design differed 
somewhat, in that the two samples did not 
include aU the same non-SD and/or LEP 
students. As indicated in the table, addi- 
tional non-SD and/or LEP students were 
included in the accommodations-permitted 
sample. 



Table A.3 



National student sample size, grades 4,8, and 12 (public and nonpublic schools combined): 1996 and 2000 

1996 2000 



Accommodations- Accommodations- 
not-permitted permitted 

sample sample 



Grade 4 

Non-SD and/or LEP students assessed 

SD and/or LEP students assessed 
without accommodations 

SD and/or LEP students assessed 
with accommodations 

Total students assessed 

Grade 8 

Non-SD and/or LEP students assessed 

SD and/or LEP students assessed 
without accommodations 

SD and/or LEP students assessed 
with accommodations 

Total students assessed 

Grade 12 

Non-SD and/or LEP students assessed 

SD and/or LEP students assessed 
without accommodations 

SD and/or LEP students assessed 
with accommodations 

Total students assessed 



6,704 


3,780* 


601 


319 


NA 


174 


7,305 


10,977 


7,122 


3.670* 


652 


364 


NA 


163 


7,774 


11,319 


7,128 


3,621* 


409 


285 


NA 


75 


7,537 


11,109 



Accommodations- Accommodations- 
not-permitted permitted 

sample sample 

15,068 



652 




750 


NA 




279 


15,720 


14,905 


16,097 


882 




798 


NA 




252 


15,787 




15,955 



14,555 



554 


607 


NA 


163 


15,109 


15,325 



*The 1996 accommodations-permitted sample included additional non-SD and/or LEP students. 

SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

NA = Not applicable. No accommodations were permitted in this sample. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table A. 4 provides a summary of the 
national school and student participation 
rates for the science assessment samples 
where accommodations were not permit- 
ted and where accommodations were 
permitted. Participation rates are presented 
for public and nonpublic schools, individu- 
ally and combined. The first rate is the 
weighted percentage of schools participat- 
ing in the assessment before substitution of 
demographically similar schools.^ This rate 
is based only on the sample of schools that 
was initially selected for the assessment. 

The numerator of this rate is the sum of 
the estimated number of students repre- 
sented by each initially selected school that 
participated in the assessment. The denomi- 
nator is the sum of the estimated number 
of students represented by each of the 
initially selected schools that had eligible 
students enrolled. 

The second school participation rate is 
the weighted participation rate after substi- 
tution. The numerator of this rate is the 
sum of the estimated number of students 
represented by each of the participating 
schools, whether originally selected or 
selected as a substitute for a school that 
chose not to participate. The denominator 
is the sum of the estimated number of 
students represented by each of the initially 
selected schools that had eligible students 
enrolled (this is the same as that for the 
weighted participation rate for the sample 
of schools before substitution). The de- 
nominator for these two rates is an estimate 



of the number of students eligible for the 
assessment, from all schools in the nation 
with eligible students enrolled. Because of 
the common denominators, the weighted 
participation rate after substitution is at 
least as great as the weighted participation 
rate before substitution. 

Also presented in table A. 4 are weighted 
student participation rates. The numerator 
of this rate is the sum across all students 
assessed (in either an initial session or a 
makeup session) of the number of students 
that each represents. The denominator of 
this rate is the sum of the number of 
students represented in the sample, across 
all eligible sampled students in participating 
schools. The overall participation rate is 
calculated as the product of the weighted 
percentage of school participation before 
(or after) substitution, and the weighted 
percentage of student participation after 
makeup sessions. 

For the grade 12 national sample, where 
school and student response rates did not 
meet NCES standards, an extensive analysis 
was conducted that examined, among 
other factors, the potential for nonresponse 
bias at both the school and student level. 

No evidence of any significant potential for 
either school or student nonresponse bias 
was found. Results of these analyses, as well 
as nonresponse bias analyses for the grade 
4 and grade 8 national samples, wiU be 
included in the forthcoming NAEP 2000 
Technical Report. 



6 The initial base sampling weights were used in weighting the percentages of participating schools and students. An 
attempt was made to preselect (before field processes began) a maximum of two substitute schools for each 
sampled public school (one in-district and one out-of-district) and each sampled Catholic school, and one for 
each sampled nonpublic school (other than Catholic). To minimize bias, a substitute school resembled the original 
selection as much as possible on affiliation, estimated number of grade-eligible students, and minority composition. 
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Table A.4 



National school and student participation rates for public schools, nonpublic schools, and public 
and nonpublic schools combined, grades 4, 8, and 12: 2000 



Weighted school participation 



Samples where accommodations 
were not permitted 



Samples where accommodations 
were permitted 











Student participation 


Overall participation rate 


Student participation 


Overall participation rate 




Percentage 

before 


Percentage 

after 


Total 

number 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 


Grade 4 


substitution 


substitution 


of schools 


participation 


assessed 


substitution 


substitution 


participation 


assessed 


substitution 


substitution 


Public 


85 


88 


414 


96 


9,144 


81 


85 


95 


9,484 


81 


84 


Nonpublic 


85 


88 


363 


96 


6,576 


82 


85 


96 


6,613 


82 


85 


Combined 


85 


88 


111 


96 


15,720 


81 


85 


96 


16,097 


81 


85 


Grade 8 
























Public 


83 


85 


385 


92 


9,443 


76 


78 


91 


9,617 


76 


78 


Nonpublic 


81 


84 


366 


96 


6,344 


77 


81 


96 


6,338 


77 


81 


Combined 


82 


85 


751 


92 


15,787 


76 


78 


92 


15,955 


76 


78 


Grade 12 
























Public 


78 


82 


243 


75 


8,562 


58 


61 


75 


8,727 


58 


61 


Nonpublic 


73 


80 


307 


89 


6,547 


65 


71 


89 


6,598 


65 


71 


Combined 


77 


82 


550 


76 


15,109 


59 


62 


76 


15,325 


59 


62 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 



State Samples 

The results of the 2000 state assessment 
program in science provided in this report 
are based on state-level samples of fourth- 
and eighth-grade public school students 
independent of the national samples. The 
samples were selected using a two-stage 
sample design that first selected schools 
within participating jurisdictions and 
then students within schools. As with the 
national samples, the jurisdiction samples 



were weighted to allow for valid inferences 
about the populations of interest. Tables 
A. 5a and A. 5b contain the unweighted 
number of participating schools and stu- 
dents, as well as weighted school and 
student participation rates for state samples 
where accommodations were not permit- 
ted and where accommodations were 
permitted. Participation rates for the states 
were calculated the same way rates were 
computed for the nation. 



O 

ERIC 



PPENDIX A 



SCIENCE REPORT CARD 



168 




Table A.5a 



State school and student participation rates, grade 4 (public schools only): 2000 



Weighted school participation 



Samples where accommodations 
were not permitted 



Samples where accommodations 
were permitted 















Overall participation rate 






Overall participation rate 




Percentage 

before 


Percentage 

after 


Total 

number 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 




substitution 


substitution 


of schools 


participation 


assessed 


substitution 


substitution 


participation 


assessed 


substitution 


substitution 


Nation 


85 


88 


414 


96 


9,144 


81 


85 


95 


9,484 


81 


84 


Alabama 


87 


94 


109 


96 


2,526 


83 


91 


96 


2,552 


83 


91 


Arizona 


87 


87 


95 


93 


2,080 


81 


81 


93 


2,068 


81 


81 


Arkansas 


85 


85 


93 


95 


2,175 


81 


81 


95 


2,214 


81 


81 


California 


76 


76 


81 


94 


1,682 


72 


72 


94 


1,714 


71 


71 


Connecticut 


100 


100 


107 


96 


2,493 


96 


96 


95 


2,550 


95 


95 


Georgia 


99 


99 


107 


95 


2,640 


94 


94 


94 


2,687 


94 


94 


Hawaii 


98 


98 


106 


95 


2,425 


94 


94 


95 


2,439 


93 


93 


Idaho ^ 


75 


75 


78 


95 


1,717 


71 


71 


95 


1,750 


71 


71 


Illinois t 


73 


73 


77 


95 


1,596 


70 


70 


95 


1,671 


70 


70 


Indiana^ 


70 


70 


78 


95 


1,812 


66 


66 


95 


1,870 


67 


67 


lowai^ 


71 


71 


89 


96 


1,887 


68 


68 


95 


1,951 


67 


67 


Kentucky 


92 


94 


105 


95 


2,248 


87 


89 


95 


2,311 


87 


89 


Louisiana 


100 


100 


108 


95 


2,452 


95 


95 


, 95 


2,538 


95 


95 


Maine 


85 


85 


107 


95 


2,094 


81 


81 


94 


2,184 


81 


81 


Maryland 


100 


100 


no 


95 


2,648 


95 


95 


94 


2,737 


94 


94 


Massachusetts 


99 


99 


106 


95 


2,274 


94 


94 


95 


2,351 


94 


94 


Michigan^ 


71 


83 


83 


94 


1,875 


67 


78 


94 


1,922 


67 


78 


Minnesota ^ 


83 


83 


78 


95 


1,853 


79 


79 


95 


1,894 


78 


78 


Mississippi 


98 


98 


106 


95 


2,776 


93 


93 


95 


2,799 


93 


93 


Missouri 


96 


96 


103 


95 


2,367 


91 


91 


94 


2,473 


91 


91 


Montana ^ 


76 


77 


67 


95 


1,176 


72 


74 


95 


1,201 


72 


74 


Nebraska 


96 


96 


73 


94 


1,289 


90 


90 


95 


1,315 


91 


91 


Nevada 


100 


100 


109 


94 


2,526 


94 


94 


94 


2,619 


94 


94 


New Mexico 


93 


93 


98 


94 


1,895 


87 


87 


94 


1,999 


87 


87 


New York ^ 


72 


72 


79 


93 


1,764 


67 


67 


93 


1,848 


67 


67 


North Carolina 


100 


100 


108 


95 


2,374 


95 


95 


95 


2,482 


95 


95 


North Dakota 


89 


89 


129 


96 


2,338 


86 


86 


97 


2,400 


86 


86 


Ohio^ 


82 


82 


85 


93 


1,887 


76 


76 


93 


1,922 


76 


76 


Oklahoma 


99 


99 


120 


95 


2,377 


93 


93 


94 


2,475 


93 


93 


Oregon ^ 


73 


74 


79 


94 


1,625 


69 


70 


95 


1,686 


69 


70 


Rhode Island 


100 


100 


no 


95 


2,395 


95 


95 


95 


2,500 


95 


95 


South Carolina 


97 


97 


103 


96 


2,448 


93 


93 


96 


2,495 


93 


93 


Tennessee 


97 


97 


105 


95 


2,496 


92 


92 


95 


2,522 


92 


92 


Texas 


97 


99 


100 


96 


2,125 


93 


95 


96 


2,229 


93 


95 


Utah 


100 


100 


no 


95 


2,652 


95 


95 


95 


2,694 


95 


95 


Vermont^ 


75 


75 


66 


95 


1,237 


71 


71 


95 


1,312 


71 


71 


Virginia 


100 


100 


108 


96 


2,502 


96 


96 


96 


2,615 


96 


96 


West Virginia 


100 


100 


126 


95 


2,522 


95 


95 


95 


2,639 


95 


95 


Wisconsin ^ 


65 


67 


69 


95 


1,393 


62 


64 


96 


1,474 


62 


64 


Wyoming 


100 


100 


93 


95 


1,745 


95 


95 


95 


1,821 


95 


95 


Other Jurisdictions 
American Samoa 


100 


100 


17 


93 


453 


93 


93 


93 


475 


93 


93 


DDESS 


100 


100 


39 


95 


1,295 


95 


95 


96 


1,300 


96 


96 


DoDDS 


100 


100 


84 


95 


2,790 


95 


95 


96 


2,825 


96 


96 


Guam 


96 


96 


23 


95 


996 


90 


90 


95 


1,064 


91 


91 


Virgin Islands 


100 


100 


22 


96 


690 


96 


96 


96 


698 


96 


96 
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^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table A.5b 




State school and student participation rates, grade 8 (public schools only): 2000 



Weighted school participation 



Samples where accommodations 
were not permitted 



Samples where accommodations 
were permitted 















Overall participation rate 






Overall participation rate 




Percentage 

before 


Percentage 

after 


Total 

number 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 




substitution 


substitution 


of schools 


participation 


assessed 


substitution 


substitution 


participation 


assessed 


substitution 


substitution 


Nation 


83 


85 


385 


92 


9,443 


76 


78 


91 


9,617 


76 


78 


Alabama 


82 


92 


102 


94 


2,400 


77 


86 


93 


2,382 


77 


86 


Arizona ^ 


76 


76 


80 


91 


1,783 


69 


69 


91 


1,822 


69 


69 


Arkansas 


87 


87 


92 


92 


2,115 


80 


80 


92 


2,140 


80 


80 


California ^ 


72 


72 


76 


93 


1,650 


67 


67 


93 


1,723 


67 


67 


Connecticut 


100 


100 


104 


91 


2,506 


91 


91 


91 


2,551 


91 


91 


Georgia 


99 


99 


102 


92 


2,550 


91 


91 


92 


2,578 


91 


91 


Hawaii 


91 


91 


50 


90 


2,268 


82 


82 


91 


2,285 


83 


83 


Idaho ^ 


78 


78 


63 


93 


1,973 


73 


73 


93 


2,003 


73 


73 


Illinois'^ 


75 


75 


80 


94 


1,753 


70 


70 


93 


1,808 


70 


70 


Indiana^ 


73 


73 


76 


93 


1,878 


68 


68 


93 


1,904 


68 


68 


Kentucky 


94 


95 


96 


94 


2,303 


89 


90 


94 


2,383 


89 


90 


Louisiana 


100 


100 


104 


91 


2,373 


91 


91 


90 


2,393 


90 


90 


Maine‘S 


83 


85 


86 


94 


2,156 


78 


79 


94 


2,254 


78 


79 


Maryland 


97 


97 


103 : 


89 


2,336 


86 


86 


89 


2,434 


87 


87 


Massachusetts 


99 


99 


99 


93 


2,277 


92 


92 


92 


2,389 


91 


91 


Michigan + 


72 


81 


86 


91 


2,024 


65 


74 


91 


2,047 


65 


73 


Minnesota ^ 


73 


73 


59 


92 


1,435 


68 


68 


92 


1,458 


68 


68 


Mississippi 


98 


98 


101 


93 


2,495 


91 


91 


93 


2,514 


91 


91 


Missouri 


92 


94 


104 


93 


2,320 


86 


88 


93 


2,415 


86 


87 


Montana ^ 


73 


74 


62 


92 


1,692 


68 


69 


93 


1,745 


68 


69 


Nebraska 


98 


98 


87 


91 


1,898 


90 


90 


90 


1,863 


89 


89 


Nevada 


100 


100 


64 


92 


2,694 


92 


92 


91 


2,733 


91 


91 


New Mexico 


91 


91 


85 


89 


1,903 


81 


81 


89 


1,981 


81 


82 


New York ^ 


71 


71 


74 


89 


1,616 


63 


63 


89 


1,697 


63 


63 


North Carolina 


98 


98 


103 


92 


2,342 


90 


90 


91 


2,452 


90 


90 


North Dakota 


91 


91 


93 


93 


2,194 


84 


84 


92 


2,221 


84 


84 


Ohio 


91 


91 


88 


92 


2,122 


83 


83 


91 


2,169 


83 


83 


Oklahoma 


100 


100 


114 


92 


2,452 


92 


92 


93 


2,515 


93 


93 


Oregon ^ 


74 


74 


78 


90 


1,751 


67 


67 


90 


1,780 


67 


67 


Rhode Island 


100 


100 


52 


91 


2,360 


91 


91 


90 


2,440 


90 


90 


South Carolina 


91 


92 


95 


93 


2,298 


85 


86 


93 


2,336 


85 


86 


Tennessee 


90 


92 


97 


91 


2,227 


82 


83 


91 


2,257 


82 


84 


Texas 


91 


94 


100 


93 


2,302 


85 


88 


92 


2,331 


84 


87 


Utah 


100 


100 


95 


92 


2,446 


92 


92 


92 


2,475 


92 


92 


Vermont ^ 


80 


80 


74 


93 


1,966 


74 


74 


92 


2,021 


74 


74 


Virginia 


100 


100 


105 


91 


2,435 


91 


91 


90 


2,508 


90 


90 


West Virginia 


100 


100 


102 


93 


2,436 


93 


93 


92 


2,567 


92 


92 


Wisconsin ^ 


66 


75 


80 


91 


1,811 


61 


68 


91 


1,883 


60 


68 


Wyoming 


100 


100 


64 


93 


2,560 


93 


93 


93 


2,575 


93 


93 


Other Jurisdictions 
American Samoa 


96 


96 


16 


97 


445 


93 


93 


97 


471 


93 


93 


DDESS 


100 


100 


14 


94 


650 


94 


94 


95 


701 


95 


95 


DoDDS 


100 


100 


53 


94 


1,962 


94 


94 


94 


1,999 


94 


94 


Guam 


100 


100 


7 


90 


945 


90 


90 


90 


921 


90 


90 


Virgin Islands^ 


100 


100 


7 


90 


606 


90 


90 


89 


619 


89 


89 
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Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Asian/Pacific Islander Samples 

National scale score and achievement-level 
results for fourth-grade Asian/Pacific 
Islander students in 2000 are not reported. 
Table A.6 contains average science scale 
score estimates, and their standard errors, 
for the nation and for the Asian/Pacific 
Islander subgroup for the 1996 and 2000 
assessment years. In 2000, the average scale 
score for Asian/Pacific Islanders at grade 4 
\vas 8 points higher than in 1996. However, 
this cross-year difference was not statistically 
significant. 

It is important to note that aU NAEP 
results are estimates and are subject to some 
degree of sampling variability. If different 
samples of schools or students had been 
obtained, results for some subgroups would 
be higher than reported here and some 
would be lower. In most subgroups, par- 
ticularly large subgroups or subgroups for 
which special sampling procedures are 
employed, estimates of performance are 



likely to remain similar from one sample to 
another. However, the national population 
of Asian/Pacific Islander students is small 
(about 3 percent of the national popula- 
tion), heterogeneous with respect to aca- 
demic achievement, and highly clustered in 
certain locations and schools. These factors 
are associated with large sampling variabil- 
ity in survey results and are reflected in the 
large standard errors associated with perfor- 
mance estimates for this subgroup. Further- 
more, the sampling plan for the national 
assessment does not include explicit stratifi- 
cation procedures designed to mitigate 
these factors. The occurrence of the large, 
but statistically nonsignificant, change in 
the 2000 grade 4 Asian/Pacific Islander 
results was a likely consequence of these 
factors: 1) the heterogeneous nature of the 
Asian/Pacific Islander population; 2) the 
current NAEP sampling design; and 3) the 
sample sizes that were assessed. 



Table A.6 



Average science scale scores for the Asian/Pacific Islander subgroup, grade 4 (public and nonpublic 
schools combined): 1996 and 2000 





1996 


2000 






Percentage 


Average score 


Percentage 


Average score 


All students at grade 4 


100 


150 (0.8) 


100 


150 (0.7) 


Asian/ Pacific Islander at grade 4 


3 (0.2) 


151 (3.6) 


3 (0.2) 


159 (4.1) 



NOTE: The standard errors of the estimated percentages and average scale scores appear in parentheses. 

Results are based on administration procedures that did not permit accommodations. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Standards for State 
Sample Participation and 
Reporting of Results 

In carrying out the 2000 state assessment 
program, the National Center for 
Education Statistics (NCES) estabhshed 
participation rate standards that jurisdic- 
tions were required to meet in order for 
their results to be reported. NCES also 
established additional standards that re- 



quired the annotation of pubHshed results 
for jurisdictions whose sample participation 
rates were low enough to raise concerns 
about their representativeness. The NCES 
guidelines used to report results in the state 
assessments, and the guidehnes for notation 
when there is some risk of nonresponse 
bias in the reported results, are presented in 
this section. 



Guideline 1 



The publication of NAEP results 

The conditions that will result in the publication of a jurisdiction’s results are presented below. 

Guideline 1-Publication of Public School Results 

A jurisdiction will have its public school results published in the 2000 NAEP Science results (or in other reports 
that include all state-level results) if and only if its weighted participation rate for the initial sample of public 
schools is greater than or equal to 70 percent. Similarly, a jurisdiction will receive a separate NAEP state report if 
and only if its weighted participation rate for the initial sample of public schools is greater than or equal to 70 
percent. 

Discussion: If a jurisdiction's public school participation rate for the initial sample of schools is below 70 percent, 
there is a substantial possibility that bias will be introduced into the assessment results. This possibility remains 
even after making statistical adjustments to compensate for school nonparticipation. There remains the likelihood 
that, in aggregate, the substitute schools are sufficiently dissimilar from the originals that they are replacing and 
represent too great a proportion of the population to discount such a difference. Similarly, the assumptions 
underlying the use of statistical adjustments to compensate for nonparticipation are likely to be significantly 
violated if the initial response rate falls below the 70 percent level. Guideline 1 takes this into consideration. This 
guideline is congruent with current NAGB policy, which requires that data for jurisdictions that do not have a 70 
percent before-substitution participation rate be reported “in a different format,” and with the Education 
Information Advisory Committee (EIAC) resolution, which calls for data from such jurisdictions not to be published. 



The following guidelines concerning 
school and student participation rates in 
the NAEP state assessment program were 
estabhshed to address four significant ways 
in which nonresponse bias could be intro- 
duced into the jurisdiction sample esti- 
mates. Presented on the following pages 



are the conditions that will result in a 
jurisdiction’s receiving a notation in the 
2000 reports. Note that in order for a 
jurisdiction’s results to be pubhshed with 
no notations, that jurisdiction must satisfy 
all guidelines. 
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Guideline 2 



Reporting school and student participation rates with possible bias due to school nonresponse 
Guideline 2-Notation for Overall Public School Participation Rate 

A jurisdiction that meets Guideline 1 will receive a notation if its weighted participation rate for the initial sample 
of public schools was below 85 percent, and the weighted public school participation rate after substitution was 
below 90 percent. 

Discussion: For jurisdictions that did not use substitute schools, the participation rates are based on participating 
schools from the original sample. In these situations, theNCES standards specify weighted school participation 
rates of at least 85 percent to guard against potential bias due to school nonresponse. Thus the first part of these 
guidelines, referring to the weighted school participation rate for the initial sample of schools. Is In direct 
accordance with NCES standards. 

To help ensure adequate sample representation for each jurisdiction participating In the NAEP 2000 state 
assessments, NAEP provided substitutes for nonparticipating public schools. For jurisdictions that used substitute 
schools, the assessment results will be based on the student data from all schools participating from both the 
original sample and the list of substitutes (unless both an Initial school and its substitute eventually participated, 
in which case only the data from the Initial school will be used). 

The NCES standards do not explicitly address the use of substitute schools to replace initially selected schools 
that decide not to participate in the assessment. However, considerable technical consideration was given to this 
issue. Even though the characteristics of the substitute schools were matched as closely as possible to the 
characteristics of the initially selected schools, substitution does not entirely eliminate bias due to the 
nonparticipation of Initially selected schools. Thus, for the weighted school participation rates including substitute 
schools, the guidelines were set at 90 percent. 

If a jurisdiction meets either standard (i.e., 85 percent or higher prior to substitution or 90 percent or higher 
after substitution), there will be no notation for the relevant overall school participation rate. 



Guideline 3 



Important segments of the jurisdiction’s student population that must be adequately represented 
to avoid possible nonresponse bias 




Guideline 3-Notation for Strata-Specific Public School Participation Rates 



A jurisdiction that is not already receiving a notation under Guideline 2 will receive a notation if the sample of 
public schools included a class of schools with similar characteristics that had a weighted participation rate 
(after substitution) of below 80 percent, and from which the nonparticipating schools together accounted for more 
than five percent of the jurisdiction’s total weighted sample of public schools. The classes of schools from each of 
which a jurisdiction needed minimum school participation levels were determined by degree of urbanization, 
minority enrollment, and median household income of the area in which the school is located. 

Discussion: The NCES standards specify that attention should be given to the representativeness of the sample 
coverage. Thus, if some important segment of the jurisdiction’s population is not adequately represented, it is of 
concern, regardless of the overall participation rate. 



If nonparticipating schools are concentrated within a particular class of schools, the potential for substantial 
bias remains, even if the overall level of school participation appears to be satisfactory. Nonresponse adjustment 
cells for public schools have been formed within each jurisdiction, and the schools within each cell are similar 
with respect to minority enrollment, degree of urbanization, and/or median household Income, as appropriate for 
each jurisdiction. 
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If the weighted response rate, after substitution, for a single adjustment cell falls below 80 percent, and 
more than five percent (weighted) of the sampled schools are nonparticipants from such a cell, the potential 
for nonresponse bias is too great. This guideline is based on the NCES standard for stratum-specific school 
response rates. 



Guideline 4 



Possible student nonresponse bias 

Guideline 4-Notation for Overall Student Participation Rate in Public Schools 

A jurisdiction that meets Guideline 1 will receive a notation if the weighted student response rate within partici- 
pating public schools was below 85 percent. 

Discussion: This guideline follows the NCES standard of 85 percent for overall student participation rates. The 
weighted student participation rate is based on all eligible students from initially selected or substitute schools 
who participated in the assessment in either an initial session ora makeup session. If the rate falls below 85 
percent, the potential for bias due to students’ nonresponse is too great. 



Guideline 5 



Possible nonresponse bias from inadequately represented strata 

Guideline 5-Notation for Strata-Specific Student Participation Rates in Public Schools 

A jurisdiction that is not already receiving a notation under Guideline 4 will receive a notation if the sampled 
students within participating public schools included a class of students with similar characteristics that had a 
weighted student response rate of below 80 percent, and from which the nonresponding students together 
accounted for more than 5 percent of the jurisdiction’s weighted assessable public school student sample. 
Student groups from which a jurisdiction needed minimum levels of participation were determined by the age or 
grade of the student, whether or not the student was classified as a student with a disability (SD) or of limited 
English proficiency (LEP), and thetype of assessment session (monitored or unmonitored),' as well as school level 
of urbanization, minority enrollment, and median household Income of the area In which the school is located. 

Discussion: This guideline addresses the fact that if nonparticipating students are concentrated within a 
particular class of students, the potential for substantial bias remains, even if the overall student participation 
level appears to be satisfactory. Student nonresponse adjustment cells have been formed using the school-level 
nonresponse adjustment cells, together with the student’s age and the nature of the assessment session 
(unmonitored or monitored). 

If the weighted response rate for a single adjustment cell falls below 80 percent, and more than five percent 
(weighted) of the invited students who do not participate in the assessment are from such a cell, the potential 
for nonresponse bias is too great. This guideline is based on the NCES standard for stratum-specific student 
response rates. 



^ In the state assessments, 25 percent of the administration sessions were observed by quality control monitors. 
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At both fourth- and eighth-grade, one 
state, Wisconsin, failed to meet the initial 
public school participation rate of 70 
percent and, at eighth grade, the Virgin 
Islands failed to meet this standard. Results 
for these jurisdictions are not included 
with the findings reported for the state 
NAEP 2000 science assessment. 

At grade 4, there were 12 jurisdictions 
(California, Idaho, Illinois, Indiana, Iowa, 
Michigan, Minnesota, Montana, New York, 
Ohio, Oregon, andVermont) that failed to 
meet the required weighted participation 
rate of 85 percent for the initial sample of 
schools and their weighted school sample 
rate after substitution was below 90 per- 
cent. At grade 8, 12 jurisdictions (Arizona, 
California, Idaho, Illinois, Indiana, Maine, 
Michigan, Minnesota, Montana, New York, 
Oregon, andVermont) failed to meet this 
guideline as well. At grade 4, Maine failed 
to meet Guideline 3 indicating that the 
sample of public schools included a class of 
schools with similar characteristics that had 
a weighted participation rate (after substi- 
tution) of below 80 percent, and from 
which the nonparticipating schools to- 
gether accounted for more than 5 percent 
of the jurisdictions total weighted sample 
of public schools. Results for each of these 
states at the appropriate grade level are 
shown with a notation indicating possible 
bias related to nonresponse. 

Students with Disabilities (SD) 
and/or Limited English Proficient 
(LEP) Students 

It is NAEP s intent to assess all selected 
students from the target population. There- 
fore, every effort is made to ensure that all 



selected students who are capable of 
participating in the assessment are assessed. 
Some students sampled for participation in 
NAEP can be excluded from the sample 
according to carefully defined criteria. 

These criteria were revised in 1996 to 
communicate more clearly a presumption 
of inclusion except under special circum- 
stances. According to these criteria, students 
with Individualized Education Programs 
(lEPs) were to be included in the NAEP 
assessment except in the following cases: 

1) The school’s lEP team determined that 
the student could not participate, OR, 

2) The student’s cognitive functioning was 
so severely impaired that she or he could 
not participate, OR, 

3) The student’s lEP required that the 
student had to be tested with an accom- 
modation or adaptation and that the 
student could not demonstrate his or her 
knowledge without that accommodation.^ 

All LEP students receiving academic 
instruction in English for three years or 
more were to be included in the assess- 
ment. Those LEP students receiving in- 
struction in English for fewer than three 
years were to be included unless school 
staff judged them to be incapable of par- 
ticipating in the assessment in English. 

Participation of SD and/or LEP 
Students in the Two NAEP Samples 

Testing all sampled students is the best way 
for NAEP to ensure that the statistics 
generated by the assessment are as repre- 
sentative as possible of the performance of 
the entire national population and the 
populations of participating jurisdictions. 
However, all groups of students include 



^ As described in the following section, a second sample in the 1996 national and the 2000 national and state 
assessments was assessed that included students who required and were provided with accommodations. 
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certain proportions that cannot be tested in 
large-scale assessments (such as students 
who have profound mental disabihties), or 
who can only be tested through the use of 
‘‘accommodations” such as extra time, one- 
on-one administration, or use of magnify- 
ing equipment. Some students with dis- 
abilities and some LEP students cannot 
show on a test what they know and can do 
unless they are provided accommodations. 
When such accommodations are not 
allowed, students requiring such adjust- 
ments are often excluded from large-scale 
assessments such as NAEP. This phenom- 
enon has become more common in the last 
decade and gained momentum with the 
passage of the Individuals with Disabilities 
Education Act (IDEA), which led schools 
and states to identify increasing proportions 
of students as needing accommodations on 
assessments to best show what they know 
and can do.^ Furthermore, Section 504 of 
the Rehabilitation Act of 1973 requires 
that, when students with disabilities are 
tested, schools must provide them with 
appropriate accommodations so that the 
test results accurately reflect what the 
students know and are able to do.^^ In 
addition, as the proportion of English 
language learners in the population has 
increased, some states have started offering 
accommodations, such as translated ver- 
sions of assessments or the use of bihngual 
dictionaries as part of assessments. 

Before 1996, NAEP did not allow any 
testing under nonstandard conditions (i.e., 
accommodations were not permitted). At 
that time, NAEP samples were able to 



include almost all sampled students in 
“standard” assessment sessions. However, as 
the influence of IDEA grew more wide- 
spread, the failure to provide accommoda- 
tions led to increasing levels of exclusion in 
the assessment. Such increases posed two 
threats to the program: 1) they threatened 
the stability of trend lines (because exclud- 
ing more students in one year than the 
next might lead to apparent rather than real 
gains), and 2) they made NAEP samples 
less than optimally representative of target 
populations. 

NAEP reacted to this challenge by 
adopting a multipart strategy. It became 
clear that to ensure that NAEP samples 
were as inclusive as possible, the program 
had to move toward allowing the same 
assessment accommodations that were 
afforded students in state and district 
testing programs. However, allowing 
accommodations represents a change in 
testing conditions that may affect trend. 
Therefore, beginning with the 1996 na- 
tional assessments and the 1998 state 
assessments, NAEP has assessed a series of 
parallel samples of students. In one set of 
samples, testing accommodations were not 
permitted; this has allowed NAEP to 
maintain the measurement of achievement 
trends on an assessment that was, throughout 
its existence, administered under common 
conditions. In addition to the samples 
where accommodations were not permit- 
ted, parallel samples in which accommoda- 
tions were permitted were also assessed. By 
having two overlapping samples and two 
sets of related data points, NAEP could 



9 Office of Special Education Programs. (1997). Nineteenth annual report to Congress on the implementation of the 
individuals with disabilities education act. Washington, DC: U. S. Department of Education. 

Section 504 of the Rehabilitation Act of 1973 is a civil rights law designed to prohibit discrimination on the basis 
of disability in programs and activities, including education, that receive federal financial assistance. 
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meet two core program goals." First, data 
trends could be maintained. Second, paral- 
lel trend lines could be set in ways that 
ensure that in future years the program will 
be able to use the most inclusive practices 
possible and mirror the procedures used by 
most state and district assessments. Begin- 
ning in 2002, NAEP will use only the 
more inclusive samples in which assessment 
accommodations are permitted. 

In science, national and state data from 
1996 and 2000 are reported for the sample 
in which accommodations were not 
permitted. National data for the second 
sample, in which accommodations were 
permitted, are reported at all grades for 
1996 and 2000. State data on this more 
inclusive sample are reported for 2000 only. 

In order to make it possible to evaluate 
both the impact of increasing exclusion 
rates in some jurisdictions and differences 
between jurisdictions, complete data on 
exclusion in both assessment years are 
included in this appendix. Since the exclu- 
sion rates may affect trend measurement 
within a jurisdiction, readers should con- 
sider the magnitude of exclusion rate 
changes when interpreting score changes in 
jurisdictions. In addition, different rates of 
exclusion may influence the meaning of 
state comparisons. Thus, exclusion data 
should be reviewed in this context as well. 

Participation rates across the assessment 
years for students with disabilities (SD) 
and/ or Hmited EngUsh proficient (LEP) 
students for the national sample where 
accommodations were not permitted are 
presented in table A. 7. The data in this table 
include the percentages of students identi- 
fied as SD and/or LEP, the percentage of 
students excluded, and the percentage of 



assessed SD and/or LEP students. Tables 
A. 8a and A. 8b show similar information by 
jurisdiction for grades 4 and 8 (only 2000 
data are presented for grade 4 since there 
was no fourth-grade state science assess- 
ment in 1996). Participation rates for the 
national sample where accommodations 
were permitted are presented in table A.9, 
and state results where accommodations 
were permitted are shown in tables A. 10a 
and A. 10b. The data in these tables include 
the percentages of students identified as SD 
and/or LEP, the percentage of students 
excluded, the percentage of assessed SD and/ 
or LEP students, the percentage assessed 
without accommodations, and the percentage 
assessed with accommodations. Expanded 
state-level data are available on the NAEP 
web site (http://nces.ed.gov/ 
nationsreportcard) that break out these 
percentages for SD students and LEP 
students separately. 

In the 2000 accommodations-not- 
permitted national sample, 7 percent of 
students at grades 4 and 8 and 4 percent of 
students at grade 12 were excluded from 
the assessment. The comparable percentages 
in the 2000 accommodations-permitted 
national sample were 4 percent at grade 4, 

3 percent at grade 8, and 2 percent at grade 
12. This comparison would suggest that 
allowing accommodations did help to 
decrease the percentage of students ex- 
cluded firom the assessment. A similar 
pattern is evident in the various jurisdic- 
tions that participated in the 2000 state 
assessment. Across the jurisdictions, the 
percentage of students excluded in the 
accommodations-not-permitted sample 
ranged from 4 to 15 percent at grade 4, and 
from 4 to 14 percent at grade 8. In the 
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The two samples are described as “overlapping” because in 2000 the same group ofnon-SD and/or LEP students 
were included in both samples. In 1996, all of the non-SD and/or LEP students in the sample that did not permit 
accommodations were included in the analysis of results for the sample that did permit accommodations, with the 
inclusion of additional non-SD and/or LEP students selected for the accommodations-permitted sample only. 
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Table A.7 



Percentage of students identified as SD and/ or LEP where accommodations were not permitted 
(public and nonpublic schools combined): 1996 and 2000 
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1996 




2000 




Number of 


Weighted percentage 


Number of 


Weighted percentage 




students 

sampled 


of students 


students 

sampled 


of students 


Grade 4 

SD and/or LEP students 










Identified 


1,357 


16 


1,248 


14 


Excluded 


756 


8 


596 


7 


Assessed 


601 


7 


652 


7 


SD students only 










Identified 


773 


11 


782 


10 


Excluded 


425 


6 


453 


6 


Assessed 


348 


5 


329 


4 


LEP students only 










Identified 


654 


5 


557 


5 


Excluded 


393 


3 


225 


2 


Assessed 


261 


2 


332 


3 


Grade 8 

SD and/or LEP students 










Identified 


1,078 


12 


1,728 


14 


Excluded 


426 


4 


846 


7 


Assessed 


652 


7 


882 


8 


SD students only 










Identified 


763 


10 


1,306 


12 


Excluded 


314 


4 


711 


6 


Assessed 


449 


6 


595 


6 


LEP students only 










Identified 


373 


3 


530 


4 


Excluded 


156 


1 


217 


1 


Assessed 


217 


2 


313 


2 


Grade 12 

SD and/or LEP students 










Identified 


834 


8 


1,066 


9 


Excluded 


425 


3 


512 


4 


Assessed 


409 


4 


554 


5 


SD students only 










Identified 


530 


5 


843 


8 


Excluded 


321 


3 


449 


4 


Assessed 


209 


3 


394 


4 


LEP students only 










Identified 


340 


3 


282 


2 


Excluded 


136 


1 


111 


1 


Assessed 


204 


2 


171 


1 



SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

NOTE: Within each grade level, the combined SD and/or LEP portion of the table is not a sum of the separate SD and LEP portions because some students were 
identified as both SD and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion. 

Within each portion of the table, percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table A.8a 



State percentage of students identified as SD and/or LEP where accommodations were not 
permitted, grade 4 (public schools only): 2000 





Identified 


Excluded 


Assessed 


Nation 


16 


8 


8 


Alabama 


12 


6 


7 


Arizona 


24 


11 


12 


Arkansas 


13 


6 


6 


California ^ 


33 


11 


22 


Connecticut 


15 


10 


5 


Georgia 


11 


8 


4 


Hawaii 


19 


9 


10 


Idaho ^ 


16 


6 


10 


Illinois ^ 


16 


9 


7 


Indiana ^ 


12 


7 


5 


Iowa ^ 


14 


10 


5 


Kentucky 


12 


8 


4 


Louisiana 


16 


8 


8 


Maine ^ 


18 


11 


7 


Maryland 


13 


9 


3 


Massachusetts 


20 


11 


9 


Michigan ^ 


11 


9 


2 


Minnesota ^ 


16 


7 


9 


Mississippi 


6 


4 


2 


Missouri 


15 


10 


5 


Montana ^ 


13 


5 


7 


Nebraska 


16 


6 


11 


Nevada 


20 


11 


9 


New Mexico 


30 


13 


17 


New York ^ 


17 


13 


4 


North Carolina 


17 


14 


2 


North Dakota 


14 


6 


7 


Ohio ^ 


12 


10 


2 


Oklahoma 


20 


10 


10 


Oregon ^ 


18 


8 


10 


Rhode Island 


23 


12 


11 


South Carolina 


17 


8 


9 


Tennessee 


11 


4 


7 


Texas 


26 


15 


11 


Utah 


14 


7 


7 


Vermont ^ 


15 


11 


5 


Virginia 


15 


10 


5 


West Virginia 


13 


10 


3 


Wisconsin ^ 


20 


13 


7 


Wyoming 

Other Jurisdictions 


14 


6 


8 


American Samoa 


17 


15 


2 


DDESS 


11 


7 


4 


DoDDS 


11 


5 


6 


Guam 


26 


10 


17 


Virgin Islands 


7 


5 


2 



^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table A.8b 



State percentage of students identified as SD and/ or LEP where accommodations were not 
permitted, grade 8 (public schools only): 1996 and 2000 







1996 






2000 






Identified 


Excluded 


Assessed 


Identified 


Excluded 


Assessed 


Nation 


13 


5 


8 


16 


7 


8 


Alabama 


13 


8 


5 


13 


4 


8 


Arizona ^ 


15 


6 


9 


18 


9 


9 


Arkansas 


12 


7 


5 


15 


8 


7 


California ^ 


21 


9 


12 


26 


9 


16 


Connecticut 


15 


9 


6 


14 


9 


5 


Georgia 


10 


5 


5 


11 


7 


4 


Hawaii 


13 


5 


7 


20 


8 


12 


Idaho t 


— 


— 


— 


14 


5 


9 


Illinois ^ 


— 


— 


— 


15 


11 


5 


Indiana ^ 


11 


6 


5 


11 


6 


5 


Kentucky 


9 


4 


5 


13 


9 


3 


Louisiana 


11 


6 


5 


13 


5 


8 


Maine 


13 


7 


6 


16 


9 


7 


Maryland 


12 


6 


7 


14 


10 


4 


Massachusetts 


17 


7 


10 


20 


13 


7 


Michigan ^ 


10 


5 


4 


11 


8 


3 


Minnesota ^ 


11 


4 


7 


15 


5 


10 


Mississippi 


11 


6 


5 


8 


5 


3 


Missouri 


13 


6 


7 


13 


8 


5 


Montana ^ 


9 


3 


6 


13 


6 


6 


Nebraska 


11 


4 


7 


15 


4 


11 


Nevada 


13 


9 


5 


14 


9 


6 


New Mexico 


20 


9 


11 


26 


13 


13 


New York ^ 


15 


9 


6 


18 


14 


4 


North Carolina 


10 


5 


5 


15 


12 


2 


North Dakota 


9 


2 


7 


13 


4 


9 


Ohio 


— 


— 


— 


11 


8 


3 


Oklahoma 


— 


— 


— 


14 


8 


7 


Oregon ^ 


12 


5 


7 


17 


6 


11 


Rhode Island 


17 


7 


10 


19 


10 


9 


South Carolina 


10 


7 


4 


14 


8 


6 


Tennessee 


12 


4 


8 


14 


6 


8 


Texas 


17 


8 


9 


19 


9 


11 


Utah 


9 


4 


5 


12 


6 


6 


Vermont ^ 


14 


6 


8 


19 


11 


9 


Virginia 


12 


7 


6 


15 


10 


5 


West Virginia 


12 


7 


5 


16 


11 


4 


Wisconsin ^ 


11 


7 


4 


15 


9 


6 


Wyoming 
Other Jurisdictions 


10 


4 


6 


12 


4 


8 


American Samoa 


— 


— 


— 


15 


12 


3 


DDESS 


10 


6 


3 


15 


13 


3 


DoDDS 


8 


3 


5 


8 


4 


4 


Guam 


9 


7 


2 


17 


5 


12 
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^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table A.9 



Percentage of students identified as SD and/or LEP where accommodations were permitted 
(public and nonpublic schools combined): 1996 and 2000 

1996 2000 





Number of 


Weighted percentage 


Number of 


weighted percentage 


Grade 4 


students sampled 


of students 


students sampled 


of students 


SD and/or LEP students Identified 


820 


16 


1,427 


16 


Excluded 


327 


6 


398 


4 


Assessed 


493 


10 


1,029 


12 


Assessed without accommodations 


319 


6 


750 


8 


Assessed with accommodations 


174 


4 


279 


4 


SD students only Identified 


496 


12 


860 


11 


Excluded 


145 


4 


257 


3 


Assessed 


351 


8 


603 


8 


Assessed without accommodations 


192 


4 


367 


5 


Assessed with accommodations 


159 


4 


236 


3 


LEP students only Identified 


370 


5 


649 


6 


Excluded 


196 


2 


193 


1 


Assessed 


174 


3 


456 


4 


Assessed without accommodations 


138 


2 


402 


4 


Assessed with accommodations 


36 


1 


54 


1 


Grade 8 

SD and/or LEP students Identified 


850 


11 


1,468 


13 


Excluded 


323 


4 


418 


3 


Assessed 


527 


7 


1,050 


9 


Assessed without accommodations 


364 


5 


798 


7 


Assessed with accommodations 


163 


3 


252 


2 


SD students only Identified 


604 


8 


1,094 


10 


Excluded 


244 


3 


354 


3 


Assessed 


360 


6 


740 


7 


Assessed without accommodations 


223 


3 


511 


5 


Assessed with accommodations 


137 


2 


229 


2 


LEP students only Identified 


283 


3 


427 


3 


Excluded 


101 


1 


88 


1 


Assessed 


182 


2 


339 


2 


Assessed without accommodations 


147 


1 


303 


2 


Assessed with accommodations 


35 


A 


36 


A 


Grade 12 

SD and/or LEP students Identified 


596 


7 


1,065 


9 


Excluded 


236 


3 


295 


2 


Assessed 


360 


5 


770 


7 


Assessed without accommodations 


285 


4 


607 


5 


Assessed with accommodations 


75 


1 


163 


2 


SD students only Identified 


395 


5 


726 


7 


Excluded 


203 


2 


257 


2 


Assessed 


192 


3 


469 


5 


Assessed without accommodations 


119 


2 


329 


3 


Assessed with accommodations 


73 


1 


140 


2 


LEP students only Identified 


228 


2 


385 


2 


Excluded 


52 


A 


75 


A 


Assessed 


176 


2 


310 


2 


Assessed without accommodations 


171 


2 


286 


2 


Assessed with accommodations 


5 


A 


24 


A 




A Percentage is between 0.0 and 0.5. 

SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

NOTE: Within each grade level, the combined SD and/or LEP portion of the table is not a sum of the separate SD and LEP portions because some students were 
identified as both SD and LEP. Such students would be counted separately in the bottom portions but counted only once in the top portion. 

Within each portion of the table, percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table A.IOa 



State percentage of students identified as SD and/or LEP where accommodations were permitted, 
grade 4 (public schools only): 2000 





Identified SD and/or LEP 

Total Excluded 


Total 


Assessed SD and/or LEP 

Assessed under Assessed 

standard with 

conditions accommodations 


All students 
assessed under 
standard 
conditions 


Nation 


18 


5 


13 


9 


4 


91 


Alabama 


12 


4 


8 


6 


3 


93 


Arizona 


24 


6 


18 


11 


7 


87 


Arkansas 


13 


4 


9 


5 


4 


92 


California ^ 


33 


5 


28 


19 


9 


86 


Connecticut 


15 


5 


10 


5 


5 


90 


Georgia 


11 


3 


8 


5 


3 


93 


Hawaii 


19 


8 


11 


9 


2 


89 


Idaho ^ 


16 


2 


13 


8 


6 


92 


Illinois ^ 


16 


4 


12 


6 


6 


90 


Indiana ^ 


12 


3 


8 


4 


4 


92 


Iowa ^ 


14 


3 


12 


4 


7 


90 


Kentucky 


12 


4 


9 


4 


5 


91 


Louisiana 


16 


2 


13 


2 


11 


86 


Maine ^ 


18 


4 


14 


5 


8 


87 


Maryland 


13 


3 


9 


4 


6 


91 


Massachusetts 


20 


4 


16 


6 


10 


87 


Michigan ^ 


11 


3 


8 


4 


3 


93 


Minnesota ^ 


16 


3 


13 


6 


7 


90 


Mississippi 


6 


2 


4 


2 


2 


95 


Missouri 


15 


1 


13 


5 


8 


90 


Montana ^ 


13 


3 


10 


5 


5 


93 


Nebraska 


16 


5 


12 


8 


3 


92 


Nevada 


20 


7 


14 


9 


5 


89 


New Mexico 


30 


6 


23 


17 


7 


87 


New York ^ 


17 


4 


12 


3 


10 


86 


North Carolina 


17 


6 


11 


4 


8 


87 


North Dakota 


14 


1 


12 


8 


4 


94 


Ohio ^ 


12 


4 


8 


3 


5 


91 


Oklahoma 


20 


4 


16 


11 


5 


91 


Oregon ^ 


18 


4 


14 


7 


7 


90 


Rhode Island 


23 


4 


19 


9 


10 


86 


South Carolina 


17 


5 


11 


7 


5 


90 


Tennessee 


11 


2 


9 


7 


2 


96 


Texas 


26 


8 


18 


14 


5 


87 


Utah 


14 


4 


10 


6 


4 


92 


Vermont ^ 


15 


3 


13 


4 


9 


88 


Virginia 


15 


5 


10 


5 


5 


90 


West Virginia 


13 


3 


10 


3 


7 


90 


Wisconsin ^ 


20 


5 


16 


6 


10 


85 


Wyoming 


14 


1 


13 


6 


7 


92 


Other Jurisdictions 
American Samoa 


17 


7 


10 


10 


0 


93 


DDESS 


11 


5 


7 


3 


4 


92 


DoDDS 


11 


2 


8 


4 


4 


94 


Guam 


26 


6 


20 


15 


6 


88 


Virgin Islands 


7 


4 


2 


2 


0 


96 



^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table A.IOb 



State percentage of students identified as SD and/or LEP where accommodations were permitted, 



grade 8 (public schools only): 2000 





Identified SD and/or LEP I 




Assessed SD and/or LEP 


All students 




Total 


Excluded 


Total 


Assessed under 
standard 
conditions 


Assessed 

with 

accommodations 


assessed under 
standard 
conditions 


Nation 


14 


4 


10 


7 


3 


93 


Alabama 


13 


5 


7 


7 


1 


94 


Arizona ^ 


18 


4 


14 


9 


5 


92 


Arkansas 


15 


5 


10 


7 


3 


92 


California ^ 


26 


4 


22 


18 


4 


92 


Connecticut 


14 


6 


8 


6 


3 


91 


Georgia 


11 


4 


7 


4 


3 


93 


Hawaii 


20 


5 


15 


13 


2 


93 


Idaho ^ 


14 


3 


11 


8 


3 


94 


Illinois ^ 


15 


5 


10 


6 


4 


90 


Indiana ^ 


11 


3 


9 


5 


4 


93 


Kentucky 


13 


3 


10 


5 


5 


92 


Louisiana 


13 


3 


10 


6 


5 


93 


Maine ^ 


16 


2 


13 


7 


6 


91 


Maryland 


14 


3 


11 


7 


4 


93 


Massachusetts 


20 


4 


16 


8 


8 


88 


Michigan ^ 


11 


5 


6 


4 


2 


93 


Minnesota ^ 


15 


4 


11 


10 


2 


95 


Mississippi 


8 


4 


4 


2 


1 


94 


Missouri 


13 


2 


11 


6 


5 


93 


Montana ^ 


13 


1 


11 


8 


3 


95 


Nebraska 


15 


4 


11 


10 


1 


95 


Nevada 


14 


4 


10 


7 


3 


93 


New Mexico 


26 


6 


20 


18 


3 


91 


New York ^ 


18 


7 


11 


3 


8 


85 


North Carolina 


15 


5 


10 


4 


5 


90 


North Dakota 


13 


1 


12 


8 


4 


94 


Ohio 


11 


4 


8 


4 


4 


92 


Oklahoma 


14 


4 


11 


9 


2 


95 


Oregon ^ 


17 


4 


13 


9 


4 


92 


Rhode Island 


19 


4 


15 


12 


3 


93 


South Carolina 


14 


6 


8 


7 


1 


93 


Tennessee 


14 


4 


10 


10 


1 


95 


Texas 


19 


6 


13 


11 


2 


92 


Utah 


12 


3 


9 


6 


3 


94 


Vermont ^ 


19 


3 


17 


10 


6 


91 


Virginia 


15 


5 


10 


5 


5 


89 


West Virginia 


16 


3 


13 


5 


8 


89 


Wisconsin ^ 


15 


4 


11 


7 


5 


92 


Wyoming 


12 


1 


11 


8 


3 


96 


Other Jurisdictions 

American Samoa 


15 


3 


12 


10 


2 


96 


DDESS 


15 


2 


13 


8 


5 


93 


DoDDS 


8 


1 


7 


5 


2 


97 


Guam 


17 


9 


8 


4 


4 


87 



^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

SD = Students with Disabilities. 

LEP = Limited-English-Proficient students. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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accommodations-permitted sample, the 
percentages of students excluded ranged 
from 1 to 8 percent at grade 4, and from 
1 to 9 percent at grade 8. As with the 
national exclusion rates, most states and 
jurisdictions excluded a smaller percentage 
of students when accommodations were 
permitted. 

Investigating the Effects of Exclusion 
Rates on Assessment Results 

As indicated by the data in the previous 
section, exclusion rates have tended to 
increase across assessment years in the 
samples that did not permit accommoda- 
tions, particularly within certain states. In 
considering the effects of exclusion rates 
on assessment results, at least two major 
issues become evident. First, if exclusion 
rates vary substantially across assessment 
years, then the ability to report trends (i.e., 
compare results between years) may be 
threatened by the fact that the results from 
different years are based on different 
proportions of the population. Second, the 
variation in exclusion rates among states 
and jurisdictions may threaten the com- 
parison of state-by-state results within a 
given year, again because the results for 
different states or jurisdictions are based on 
different proportions of the populations. 

As a consequence, NCES investigated 
the possibility of estabhshing criteria for 
including cautionary notations based on 
excessive or increased exclusion rates 
(similar to those based on overall participa- 
tion rates) in the reporting of national and 
state-by-state results. This investigation, 
however, did not reveal a consistent rela- 
tionship between levels of exclusion, or 
degrees of change in inclusion rates, and 
overall results. There were several reasons 
for this. 



First of all, real demographic differences 
influence exclusion rates in states and, thus, 
some differences may be unavoidable. 
Second, program research conducted by 
NCES and Educational Testing Service 
(ETS) was unable to identify a particular 
level of exclusion increase that seemed to 
affect scores. Third, since excluded students 
were not tested, NAEP has no direct 
information about how those students 
would have done had they been tested. 
Given these realities and uncertainties, the 
best approach seemed to be to supply all 
data about student exclusion and allow 
readers to consider it as they interpret the 
achievement data. However, it is important 
to remember that the main solutions to this 
issue he not in flagging results, but in 
ensuring that all sampled students partici- 
pate in assessments. The new, more inclu- 
sive samples that will become NAEP’s main 
samples in 2002 are intended to accomplish 
this goal. 

The move to more inclusive samples, 
however, wiU not be a perfect solution. For 
example, even within the context of the 
samples in which accommodations are 
permitted, there is still some student 
exclusion (albeit at a far lower level, as the 
data in tables A. 8 and A.9a/b show). In 
addition, the assessment accommodations 
may not have an entirely neutral impact on 
scores. In other words, it is possible that 
changes in the percentages of students 
receiving assessment accommodations may 
influence scores. It is also possible that 
differences in state and local accommoda- 
tions pohcies will affect state comparisons. 

Because of these remaining issues, NCES 
has funded several major research studies. 
These activities have been organized 
around two distinct questions. First, as was 



O 

ERIC 



PPENOIX 



A • 



SCIENCE REPORT CARO 



184 



mentioned above, some students are ex- 
cluded from even the more inclusive 
NAEP. Therefore, NCES has funded 
research into ways excluded students might 
be included in the estimation of scores for 
overall populations. In other words, re- 
search is being conducted to investigate 
weighting procedures that might be used to 
ensure the final NAEP estimates include 
data for all students in a sampled popula- 
tion. There are two general approaches that 
have been investigated. The first is an idea 
championed by Albert Beaton of Boston 
College. Beaton recommends making a 
simple assumption about excluded students: 
he would assume that, had these students 
been tested, they would have performed 
below some predefined level (for example, 
the median score or the lowest score in the 
basic achievement range) . This statistic 
(whether median or some other level) 
would be adjusted to take account of 
excluded students. 

The second approach to obtaining full 
population estimates has been recom- 
mended by Donald McLaughlin of the 
American Institutes for Research (AIR). 

His approach involves using background 
data about excluded students to estimate 
how they, as a group, would have per- 
formed had they been assessed. This ap- 
proach is based on different and stronger 
assumptions than Beaton’s. It would have 
the advantage of allowing NAEP to con- 
tinue to report all the types of statistics 
currently in use (including average scores). 

The results from an initial examination 
of the 1996 and 2000 NAEP science data 
using McLaughhn s approach indicated that 



the reported average score gains from 1996 
to 2000 in many jurisdictions would be 
somewhat smaller if full-population esti- 
mates were used. This is apparently due to 
the increase in exclusion rates between 
years within these states. It should be noted 
that using such full-population estimates 
may not only alter the estimates of score 
gains, but may also alter the rank ordering 
of states within a given year. 

NCES has not yet judged either statisti- 
cal adjustment approach ready for opera- 
tional use. Therefore, these “fuU population 
reporting” approaches may or may not be 
used in future years. Results of the studies 
produced by McLaughlin may be obtained 
from NCES, as can copies of an Educa- 
tional Testing Service (ETS) study that 
implemented Beaton’s methodology. 

In addition to full population reporting 
research, NCES has commissioned studies 
of the impact of assessment accommoda- 
tions on overall scores. Specifically, ETS has 
conducted diflferential item functioning 
(DIF) studies of items assessed with accom- 
modation in the 1996 assessment. In 
these studies, ETS researchers found litde 
evidence that accommodations changed 
the functioning of test questions. 

Types of Accommodations Permitted 

TableA.il displays the number and the 
percentages of SD and/or LEP students 
assessed with the variety of available ac- 
commodations. It should be noted that 
students assessed with accommodations 
typically received some combination of 
accommodations. The numbers and per- 
centages presented in the table reflect only 



12 For information on DIF studies of items assessed with accommodations in the 1996 mathematics and science 
assessments, see Mazzeo,J. M., Carlson,]. E.,Voelkl, K. E., & Lutkus, A. D (1999). Increasing the participation of special 
needs students in NAEP: A report on 1996 NAEP research cJchWhes. Washington, DC: U.S. Department of Education, 
Office of Educational Research and Improvement, National Center for Education Statistics. 
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the primary accommodation provided. For 
example, students assessed in small groups 
(as compared to standard NAEP sessions 
of about 30 students) usually received 
extended time. In one-on-one administra- 



tions, students often received assistance in 
recording ansv^ers and were afforded extra 
time. Extended time was considered the 
primary accommodation only when it was 
the sole accommodation provided. 



Table A. 11 



Percentage of students in national sample identified as SD and/ or LEP by type of accommodation 
where accommodations were permitted (public and nonpublic schools combined): 1996 and 2000 



Grade 4 Grades Grade 12 

1996 2000 1996 2000 1996 2000 





Number 


Weighted 


Number 


Weighted 


Number 


Weighted 


Number 


Weighted 


Number 


Weighted 


Number 


Weighted 




of students 


percentage 


of students 


percentage of students 


percentage of students 


percentage of students 


percentage 


of students 


percentage 




sampled 


of students 


sampled 


of students 


sampled 


of students 


sampled 


of students 


sampled 


of students 


sampled 


of students 


SD and/or LEP students 


Science glossary 


0 


0.00 


37 


0.45 


13 


0.10 


13 


0.11 


0 


0.00 


2 


0.01 


Bilingual dictionary 


NA 


NA 


0 


0.00 


NA 


NA 


2 


0.01 


NA 


NA 


10 


0.11 


Glossary/dictionary 


16 


0.23 


NA 


NA 


14 


0.15 


NA 


NA 


2 


0.02 


NA 


NA 


Large-print book 


0 


0.00 


1 


0.01 


0 


0.00 


2 


0.04 


0 


0.00 


0 


0.00 


Extended time 


28 


0.69 


50 


0.56 


29 


0.47 


54 


0.35 


30 


0.32 


64 


0.51 


Read aloud 


17 


0.56 


17 


0.29 


10 


0.19 


22 


0.24 


3 


0.07 


4 


0.06 


Small group 


99 


2.37 


137 


1.69 


89 


1.66 


140 


1.54 


26 


0.35 


68 


0.93 


One-on-one 


11 


0.22 


35 


0.69 


7 


0.08 


11 


0.11 


12 


0.18 


8 


0.10 


Scribe/computer 


NA 


NA 


0 


0.00 


NA 


NA 


5 


0.08 


NA 


NA 


4 


0.03 


Other 


3 


0.07 


2 


0.01 


1 


0.01 


3 


0.04 


2 


0.02 


3 


0.04 


SD students only 


Science glossary 


0 


0.00 


1 


0.01 


0 


0.00 


0 


0.00 


0 


0.00 


0 


0.00 


Bilingual dictionary 


NA 


NA 


0 


0.00 


NA 


NA 


1 


0.00 


NA 


NA 


0 


0.00 


Glossary/dictionary 


1 


0.02 


NA 


NA 


1 


0.01 


NA 


NA 


0 


0.00 


NA 


NA 


Large-print book 


0 


0.00 


1 


0.01 


0 


0.00 


2 


0.04 


0 


0.00 


0 


0.00 


Extended time 


28 


0.69 


49 


0.55 


29 


0.47 


52 


0.34 


30 


0.32 


54 


0.47 


Read aloud 


17 


0.56 


17 


0.29 


10 


0.19 


18 


0.19 


3 


0.07 


4 


0.06 


Small group 


99 


2.37 


131 


1.64 


89 


1.66 


137 


1.52 


26 


0.35 


68 


0.93 


One-on-one 


11 


0.22 


35 


0.69 


7 


0.08 


11 


0.11 


12 


0.18 


8 


0.10 


Scribe/computer 


NA 


NA 


0 


0.00 


NA 


NA 


5 


0.08 


NA 


NA 


4 


0.03 


Other 


3 


0.07 


2 


0.01 


1 


0.01 


3 


0.04 


2 


0.02 


2 


0.03 


LEP students only 


Science glossary 


0 


0.00 


36 


0.44 


13 


0.10 


13 


0.11 


0 


0.00 


2 


0.01 


Bilingual dictionary 


NA 


NA 


0 


0.00 


NA 


NA 


2 


0.01 


NA 


NA 


10 


0.11 


Glossary/dictionary 


16 


0.23 


NA 


NA 


14 


0.15 


NA 


NA 


2 


0.02 


NA 


NA 


Large-print book 


0 


0.00 


0 


0.00 


0 


0.00 


0 


0.00 


0 


0.00 


0 


0.00 


Extended time 


3 


0.04 


1 


0.01 


2 


0.03 


3 


0.01 


2 


0.02 


10 


0.04 


Read aloud 


2 


0.08 


1 


0.00 


4 


0.06 


6 


0.06 


0 


0.00 


0 


0.00 


Small group 


14 


0.18 


16 


0.18 


2 


0.02 


11 


0.09 


1 


0.02 


0 


0.00 


One-on-one 


1 


0.01 


0 


0.00 


0 


0.00 


0 


0.00 


0 


0.00 


1 


0.00 


Scribe/computer 


NA 


NA 


0 


0.00 


NA 


NA 


0 


0.00 


NA 


NA 


0 


0.00 


Other 


0 


0.00 


0 


0.00 


0 


0.00 


1 


0.01 


0 


0.00 


1 


0.01 



SD = Students with Disabilities. LEP = Limited-English-Proficient students. 

NA = Not Applicable. Accommodation was not offered. 

NOTE: The combined SD and/or LEP portion of the table is not a sum of the separate SD and LEP portions because some students were identified as both SD 
and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Data Collection and Scoring 

The 2000 science assessment was con- 
ducted from January through March 2000, 
with some makeup sessions in early April. 

As with all NAEP assessments, data collec- 
tion for the 2000 assessment was con- 
ducted by a trained field staff. For the 
national assessment, this was accomplished 
by staff from Westat, Inc. 

For the state assessment, testing sessions 
were conducted and administered by 
employees of state and local educational 
agencies and institutions. These employees 
were carefully trained in assessment proce- 
dures by Westat. In addition, Westat em- 
ployed quality control monitors who 
observed 25 percent of the sessions in state 
assessments. 

Materials from the 2000 assessment were 
shipped to National Computer Systems, 
where trained staff evaluated the responses 
to the constructed-response questions using 
scoring rubrics or guides prepared by 
Educational Testing Service. Each con- 
structed-response question had a unique 
scoring rubric that defined the criteria 
used to evaluate students’ responses. The 
extended constructed-response questions 
were evaluated with four- and five-level 
rubrics, and many of the short constructed- 
response questions were rated according to 
three-level rubrics that permitted partial 
credit. Other short constructed-response 
questions were scored as either acceptable 
or unacceptable. 



For the 2000 science assessment, 
approximately 4.5 million constructed 
responses were scored. This number in- 
cludes rescoring to monitor inter-rater 
reliability. The within-year average percent- 
age of exact agreement for the 2000 
national reliability sample was 95 percent at 
grade 4, 96 percent at grade 8, and 96 
percent at grade 12. 

Data Analysis and IRT Scaling 

Subsequent to the professional scoring, all 
information was transcribed to the NAEP 
database at ETS. Each processing activity 
was conducted with rigorous quality 
control. After the assessment information 
was compiled in the database, the data were 
weighted according to the population 
structure. The weighting for the national 
sample reflected the probability of selection 
for each student as a result of the sampfing 
design, adjusted for nonresponse. Through 
post-stratification, the weighting assured 
that the representation of certain subpopu- 
lations corresponded to figures from the 
U.S. Census and the Current Population 
Survey. 

The procedure used for sample weight- 
ing in the state assessments is similar to that 
used in national samples. However, there 
are two important differences. First, because 
there is no oversampfing of high-minority 
schools in state samples, the weighting 
process does not need to adjust for such a 
procedure. Second, Current Population 
Survey target totals are not available or 
stable on a state-by-state basis. Therefore, 
the post-stratification process described 
above is not utiUzed in the state program. 



These procedures are described more fully in the “Weighting and Variance Estimation” section later in this 
document. For additional information about the use of weighting procedures, see the forthcoming NAEP 2000 
Technical Report. In addition, the reader may consult the NAEP 1998 Technical Report for a discussion of weighting 
procedures that are common to all NAEP assessments. 
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Analyses were then conducted to deter- 
mine the percentages of students who gave 
various responses to each cognitive and 
background question. In determining these 
percentages for the cognitive questions, a 
distinction was made between missing 
responses at the end of a block (i.e., missing 
responses subsequent to the last question 
the student answered) and missing re- 
sponses prior to the last observed response. 
Missing responses before the last observed 
response were considered intentional 
omissions. In analysis, omitted responses to 
multiple-choice items were scored as 
fractionally correct. For constructed- 
response items, omitted responses were 
placed into the lowest score category. 
Missing responses at the end of the block 
were considered “not reached” and treated 
as if the questions had not been presented 
to the student. In calculating response 
percentages for each question, only stu- 
dents classified as having been presented 
the question were included in the denomi- 
nator of the statistic. 

It is standard NAEP practice to treat all 
nonrespondents to the last question in a 
block as if they had not reached the ques- 
tion. For multiple-choice and short con- 
structed-response questions, this practice 
produces a reasonable pattern of results in 
that the proportion reaching the last 
question is not dramatically smaller than 
the proportion reaching the next-to-last 
question. However, for science blocks that 
ended with extended constructed-response 
questions, the standard practice would 



result in extremely large drops in the 
proportion of students attempting the final 
question. Therefore, for blocks ending with 
an extended constructed-response question, 
students who answered the next-to-last 
question but did not respond to the ex- 
tended constructed-response question were 
classified as having intentionally omitted 
the last question. 

Item Response Theory (IRT) was used 
to estimate average science scale scores for 
the nation and for various subgroups of 
interest within the nation. IRT models the 
probability of answering a question in a 
certain way as a mathematical function of 
proficiency or skill. The main purpose of 
IRT analysis is to provide a common scale 
on which performance can be compared 
across groups such as those defined by 
characteristics including gender and race/ 
ethnicity. 

In producing the science scales, three 
distinct IRT models were used. Multiple- 
choice questions were scaled using the 
three-parameter logistic (3PL) model; short 
constructed-response questions rated as 
acceptable or unacceptable were scaled 
using the two-parameter logistic (2PL) 
model; and short constructed-response 
questions rated according to a three-level 
rubric, as well as extended constructed- 
response questions rated on a four- or five- 
level rubric, were scaled using a General- 
ized Partial-Credit (GPC) model. Devel- 
oped by ETS and first used in 1992, the 
GPC model permits the scafing of ques- 
tions scored according to multipoint rating 
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15 Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological 
Measurement {16)2, 159-176. ^ 8 ^ 
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schemes. The model takes full advantage of 
the information available from each of the 
student response categories used for these 
more complex constructed-response 
questions. 

The science scale is composed of three 
types of questions: multiple-choice, short 
constructed-response (scored either di- 
chotomously or allowing for partial credit), 
and extended constructed response (scored 
according to a partial-credit model). One 
question about the science scales concerns 
the amount of information contributed by 
each type of question. Unfortunately, this 
question has no simple answer for the 
NAEP science assessment, due to the 
procedures used to form the composite 
science scale. The information provided by a 
given question is determined by the IRT 
model used to scale the question. It is a 
function of the item parameters and varies 
by level of science proficiency.^^ Thus, the 
answer to the query “How much informa- 
tion do the different types of questions 
provide?” will differ for each level of 
science performance.When considering 
the composite science scale, the answer is 
even more complicated. The science data 
are scaled separately by the three fields of 
science (Earth, physical, and life), resulting 
in three separate subscales at each grade. 
The composite scale is a weighted combi- 
nation of these subscales. IRT information 
functions are only strictly comparable 
when the item parameters are estimated 
together. Because the composite scale is 
based on three separate estimation runs. 



there is no direct way to compare the 
information provided by the questions on 
the composite scale. 

Because of the BIB-spiraling design used 
by NAEP, students do not receive enough 
questions about a specific topic to provide 
reliable information about individual 
performance. (For more information on 
BIB-spiraling, see “The Assessment Design” 
section earlier in this document.) Tradi- 
tional test scores for individual students, 
even those based on IRT, would lead to 
misleading estimates of population charac- 
teristics, such as subgroup means and 
percentages of students at or above a 
certain scale-score level. Consequently, 
NAEP constructs sets of plausible values 
designed to represent the distribution of 
performance in the population. A plausible 
value for an individual is not a scale score 
for that individual, but may be regarded as 
a representative value from the distribution 
of potential scale scores for all students in 
the population with similar characteristics 
and identical patterns of item response. 
Statistics describing performance on the 
NAEP science scale are based on the 
plausible values. Under the assumptions of 
the scaling models, these population esti- 
mates will be consistent, in the sense that 
the estimates approach the model-based 
population values as the sample size 
increases, which would not be the case 
for population estimates obtained by 
aggregating optimal estimates of 
individual performance.^® 




More detailed information regarding the IRT analyses used in NAEP assessments will be provided in a forthcom- 
ing technical report on the 2000 NAEP assessments. In addition, the reader may consult the NAEP 1998 Technical 
Report for a discussion of analysis procedures that are common to all NAEP assessments. 

17 Donoghue,J. R. (1994). An empirical examination of the IRT information of polytomously scored reading items 
under the generalized partial credit model. Journal of Educational Measurement (31)4, 295—311. 

18 For theoretical and empirical justification of the procedures employed, see Mislevy, R. J. (1988). Randomization- 
based inferences about latent variables from complex samples. Psychoinetrika (56)2, 177-196. 

For computational details, see the forthcoming NAEP 2000Technical Report. 
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Item Mapping Procedures 

The science performance of fourth-, 
eighth-, and twelfth-graders can be illus- 
trated by maps that position question or 
“item’’ descriptions along the NAEP 
science scale at each grade where questions 
are likely to be answered successfully by 
students. The descriptions used on these 
maps focus on the science knowledge or 
skill needed to answer the question. For 
multiple-choice questions, the description 
indicates the knowledge or skill demon- 
strated by selection of the correct option; 
for constructed-response questions, the 
description takes into account the knowl- 
edge or skill specified by the different levels 
of scoring criteria for that question. 

To map questions to particular points on 
the NAEP science scale, a response prob- 
ability convention was adopted that would 
divide those who had a higher probability 
of success from those who had a lower 
probability. Establishing a response prob- 
abihty convention has an impact on the 
mapping of the test questions onto the 
science scale. A lower boundary convention 
maps the science questions at lower points 
along the scale, and a higher boundary 
convention maps the same questions at 
higher points on the scale. The underlying 
distribution of science skills in the popula- 
tion does not change, but the choice of a 
response probability convention does have 
an impact on the proportion of the student 
population that is reported as “able to do” 
the questions on the science scales. 



There is no obvious choice of a point 
along the probabiHty scale that is clearly 
superior to any other point. If the conven- 
tion were set with a boundary at 50 per- 
cent, those above the boundary would be 
more likely to get a question right than get 
it wrong, while those below the boundary 
would be more likely to get the question 
wrong than right. Although this convention 
has some intuitive appeal, it was rejected on 
the grounds that having a 50/50 chance of 
getting the question right shows an insuffi- 
cient degree of mastery. If the convention 
were set with a boundary at 80 percent, 
students above the criterion would have a 
high probability of success with an ques- 
tion. However, many students below this 
criterion show some level of science ability 
that would be ignored by such a stringent 
criterion. In particular, those in the range 
between 50 and 80 percent correct would 
be more likely to get the question right 
than wrong, yet would not be in the group 
described as “able to do” the question. 

In a compromise between the 50 per- 
cent and the 80 percent conventions, 

NAEP has adopted two related response 
probability conventions: 74 percent for 
multiple-choice questions with four re- 
sponse options (to correct for the possibil- 
ity of answering correctly by guessing), and 
65 percent for constructed-response ques- 
tions (where guessing is not a factor). These 
probability conventions were established, in 
part, based on an intuitive judgment that 
they would provide the best picture of 
students’ science skills. 
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Some additional support for the dual 
conventions adopted by NAEP was pro- 
vided by Huynh. He examined the IRT 
information provided by items, according 
to the IRT model used in scaling NAEP 
questions. (“Information” is used here in a 
technical sense. See the forthcoming 
NAEP 2000 Technical Report for details.) 
Following Bock, Huynh decomposed the 
item information into that provided by a 
correct response [P(q) I(q)] and that pro- 
vided by an incorrect response [(1- P(q)) 
I(q)].^® Huynh showed that the item 
information provided by a correct response 
to a constructed-response item is maxi- 
mized at the point along the science scale 
at which the probability of a correct 
response is two- thirds (for multiple-choice 
items, the information provided by a 
correct response is maximized at the point 
at which the probability of getting the item 
correct is 0.74). It should be noted, how- 
ever, that maximizing the item information 
I(q), rather than the information provided 
by a correct response [P(q) I (q)], would 
imply an item mapping criterion closer to 
50 percent. 

Results are presented in terms of the 
composite science scale. However, the 
science assessment was scaled separately for 
the three fields of science at grades 4, 8, 
and 12. The composite scale is a weighted 
combination of the three subscales for the 
three fields of science. To obtain item map 
information, a procedure developed by 
Donoghue was used.^^ This method models 
the relationship between the item response 
function for the subscale and the subscale 



structure to derive the relationship be- 
tween the item score and the composite 
scale (i.e., an item response function for the 
composite scale). This item response func- 
tion is then used to derive the probability 
used in the mapping. 

Weighting and 
Variance Estimation 

A multistage sampling design was used to 
select the students who were assessed. The 
properties of a sample selected through 
such a design could be very different from 
those of a simple random sample, in which 
every student in the target population has 
an equal chance of selection and in which 
the observations from different sampled 
students can be considered to be statisti- 
cally independent of one another. There- 
fore, the properties of the sample for the 
data collection design were taken into 
account during the analysis of the assess- 
ment data. 

One way that the properties of the 
sample design were addressed was by using 
sampling weights to account for the fact 
that the probabilities of selection were not 
identical for aU students. All population and 
subpopulation characteristics based on the 
assessment data were estimated using 
sampling weights. These weights included 
adjustments for school and student 
nonresponse. 

Not only must appropriate estimates of 
population characteristics be derived, but 
appropriate measures of the degree of 
uncertainty must be obtained for those 
statistics. Two components of uncertainty 



Huynh, H. (1994, October). Some technical aspects of standard setting. Paper presented at the Joint Conference on 
Standard Setting for Large-Scale Assessment, Washington, DC. 

20 Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more latent 
categories. Psychometrika, 37, 29-51. 

21 Donoghue,]. R. (1997, March). Item mapping to a weighted composite scale. Paper presented at the annual meeting of 
the American Educational Research Association, Chicago, IL. 
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are accounted for in the variability of 
statistics based on student ability: 1) the 
uncertainty due to sampUng only a relatively 
small number of students, and 2) the 
uncertainty due to sampling only a rela- 
tively small number of cognitive questions. 
The first component accounts for the 
variability associated with the estimated 
percentages of students who had certain 
background characteristics or who answered 
a certain cognitive question correctly. 

Because NAEP uses multistage sampUng 
procedures, conventional formulas for 
estimating sampHng variabihty that assume 
simple random sampling are inappropriate. 
NAEP uses a jackknife replication proce- 
dure to estimate standard errors. The 
jackknife standard error provides a reason- 
able measure of uncertainty for any student 
information that can be observed without 
error. However, because each student 
typically responds to only a few questions 
within any field of science, the scale score 
for any single student would be imprecise. 
In this case, plausible values methodology 
can be used to describe the performance of 
groups and subgroups of students, but the 
underlying imprecision involved in this 
step adds another component of variabihty 
to statistics based on NAEP scale scores.^^ 

Typically, when the standard error is 
based on a small number of students or 
when the group of students is enrolled in a 
small number of schools, the amount of 
uncertainty associated with the estimation 
of standard errors may be quite large. 



Estimates of standard errors subject to a 
large degree of uncertainty are followed by 
the symbol to indicate that the nature 
of the sample does not allow accurate 
determination of the variabihty of the 
statistic. In such cases, the standard errors — 
and any confidence intervals or significance 
tests involving these standard errors — 
should be interpreted cautiously. Additional 
details concerning procedures for identify- 
ing such standard errors are discussed in the 
forthcoming NAEP 2000 Technical Report. 

The reader is reminded that, as with 
findings from aU surveys, NAEP results are 
subject to other kinds of error, including 
the effects of imperfect adjustment for 
student and school nonresponse and 
unknowable effects associated with the 
particular instrumentation and data 
collection methods. Nonsampling errors 
can be attributed to a number of sources — 
inabihty to obtain complete information 
about all selected schools in the sample 
(some students or schools refused to par- 
ticipate, or students participated but an- 
swered only certain questions); ambiguous 
definitions; differences in interpreting 
questions; inabihty or unwiUingness to give 
correct background information; mistakes 
in recording, coding, or scoring data; and 
other errors in coUecting, processing, 
sampling, and estimating missing data. The 
extent of nonsamphng error is difficult to 
estimate and, because of their nature, the 
impact of such errors cannot be reflected in 
the data-based estimates of uncertainty 
provided in NAEP reports. 



22 For further details, see Johnson, E. G., & Rust, K. F. (1992). Population inferences and variance estimation for 
NAEP d^t 2 . Journal of Educational Statistics (i7)2, 175-190. 
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Drawing Inferences 
from the Results 

The reported statistics are estimates and 
are therefore subject to a measure of 
uncertainty. There are two sources of such 
uncertainty. First, NAEP uses a sample of 
students rather than testing all students. 
Second, all assessments have some amount 
of uncertainty related to the fact that they 
cannot ask all questions that might be 
asked in a content area. The magnitude of 
this uncertainty is reflected in the standard 
error of each of the estimates. When the 
percentages or average scale scores of 
certain groups are compared, the standard 
error should be taken into account, and 
observed similarities or diflferences should 
not be relied on solely. Therefore, the 
comparisons are based on statistical tests 
that consider the standard errors of those 
statistics and the magnitude of the diflfer- 
ence among the averages or percentages. 

Using confidence intervals based on the 
standard errors provides a way to take into 
account the uncertainty associated with 
sample estimates and to make inferences 
about the population averages and percent- 
ages in a manner that reflects that uncer- 
tainty. An estimated sample average scale 
score plus or minus 1.96 standard errors 
approximates a 95 percent confidence 
interval for the corresponding population 
quantity. This statement means that one can 
conclude with approximately a 95 percent 
level of confidence that the average perfor- 
mance of the entire population of interest 
(e.g., all fourth-grade students in public 
and nonpublic schools) is within plus or 
minus 1.96 standard errors of the sample 
average. 



As an example, suppose that the average 
science scale score of the students in a 
particular group was 156 with a standard 
error of 1.2. A 95 percent confidence 
interval for the population quantity would 
be as follows: 

Average ± 1.96 standard errors 
156 ± 1.96x1.2 
156 ± 2.35 
(153.65,158.35) 

Thus, one can conclude with a 95 
percent level of confidence that the average 
scale score for the entire population of 
students in that group is between 153.65 
and 158.35. It should be noted that this 
example, and the examples in the following 
sections are illustrative. More precise esti- 
mates carried out to one or more decimal 
places are used in the actual analyses. 

Similar confidence intervals can be 
constructed for percentages, if the percent- 
ages are not extremely large or extremely 
small. Extreme percentages should be 
interpreted with caution. Adding or sub- 
tracting the standard errors associated with 
extreme percentages could cause the 
confidence interval to exceed 100 percent 
or go below 0 percent, resulting in num- 
bers that are not meaningful. The forth- 
coming NAEP 2000 Technical Report will 
contain a more complete discussion of 
extreme percentages. 

Analyzing Group Differences in 
Averages and Percentages 

Statistical tests determine whether the 
evidence, based on the data from the 
groups in the sample, is strong enough to 
conclude that the averages or percentages 
are actually diflferent for those groups in 
the population. If the evidence is strong 
(i.e., the diflference is statistically signifi- 
cant), the report describes the group 
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averages or percentages as being different 
(e.g., one group performed higher than or 
lower than another group), regardless of 
whether the sample averages or percentages 
appear to be approximately the same. 

The reader is cautioned to rely on the 
results of the statistical tests rather than on 
the apparent magnitude of the difference 
between sample averages or percentages 
when determining whether the sample 
differences are likely to represent actual 
differences among the groups in the popu- 
lation. 

To determine whether a real difference 
exists between the average scale scores (or 
percentages of a certain attribute) for two 
groups in the population, one needs to 
obtain an estimate of the degree of uncer- 
tainty associated with the difference be- 
tween the averages (or percentages) of 
these groups for the sample. This estimate 
of the degree of uncertainty, called the 
“standard error of the difference” between 
the groups, is obtained by taking the square 
of each group s standard error, summing 
the squared standard errors, and taking the 
square root of that sum. 

Standard Error of the Difference = 

SE... = V(SE.= + SE„^ 

Similar to how the standard error for an 
individual group average or percentage is 
used, the standard error of the difference 
can be used to help determine whether 
differences among groups in the population 
are real. The difference between the aver- 
ages or percentages of the two groups plus 
or minus 1.96 standard errors of the differ- 
ence represents an approximate 95 percent 



confidence interval. If the resulting interval 
includes zero, there is insufficient evidence 
to claim a real difference between the 
groups in the population. If the interval 
does not contain zero, the difference 
between the groups is statistically signifi- 
cant (different) at the 0.05 level. 

As an example of comparing groups, 
consider the problem of determining 
whether the average science scale score of 
group A is higher than that of group B. 
Suppose that the sample estimates of the 
average scale scores and standard errors 
were as follows: 



Group 


Average 
Scale Score 


Standard Error 


A 


218 


0.9 


B 


216 


1.1 



The difference between the estimates of 
the average scale scores of groups A and B 
is two points (218 - 2 16). The standard 
error of this difference is 

V(0.92 + l.P) = 1.4 

Thus, an approximate 95 percent confi- 
dence interval for this difference is plus or 
minus two standard errors of the difference 

2 ± 1.96 X 1.4 
2 ± 2.74 
(-0.74, 4.74) 

The value zero is within the confidence 
interval; therefore, there is insufficient 
evidence to claim that group A outper- 
formed group B. 
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Conducting Multiple Tests 

The procedures in the previous section and 
the certainty ascribed to intervals (e.g., a 95 
percent confidence interval) are based on 
statistical theory that assumes that only one 
confidence interval or test of statistical 
significance is being performed. Ho\vever, 
many different groups are being compared 
(i.e., multiple sets of confidence intervals 
are being analyzed). In sets of confidence 
intervals, statistical theory indicates that the 
certainty associated \vith the entire set of 
intervals is less than that attributable to 
each individual comparison from the set. 

To hold the significance level for the set of 
comparisons at a particular level (e.g., 0.05), 
adjustments (called “multiple comparison 
procedures’’^^) must be made to the meth- 
ods described in the previous section. One 
such procedure, the False Discovery Rate 
(FDR) procedure^"^ was used to control the 
certainty level. 

Unlike the other multiple comparison 
procedures (e.g., the Bonferroni procedure) 
that control the familywise error rate (i.e., 
the probability of making even one false 
rejection in the set of comparisons), the 
FDR procedure controls the expected 
proportion of falsely rejected hypotheses. 
Furthermore, familywise procedures are 
considered conservative for large families of 
comparisons.^^ Therefore, the FDR proce- 
dure is more suitable for multiple compari- 
sons in NAEP than other procedures. A 
detailed description of the FDR procedure 
appears in the forthcoming NAEP 2000 
Technical Report. 



To illustrate how the FDR procedure is 
used, consider the comparisons of current 
and previous years’ average science scale 
scores for the five groups presented in table 
A. 12. Note that the difference in average 
scale scores and the standard error of the 
difference are calculated in a way compa- 
rable with that of the example in the 
previous section. The test statistic shown is 
the difference in average scale scores 
divided by the standard error of the 
difference. 

The difference in average scale scores 
and its standard error can be used to find 
an approximate 95 percent confidence 
interval as in the example in the previous 
section or they can be used to identify a 
confidence percentage. In the example in 
the previous section, because an approxi- 
mate 95 percent confidence interval was 
desired, the number 1.96 was used to 
multiply the standard error of the differ- 
ence to create the approximate confidence 
interval. In the current example, the confi- 
dence interval for the test statistics is 
identified from statistical tables. Instead of 
checking to see if zero is within the 95 
percent confidence interval about the 
mean, the significance level from the 
statistical tables can be directly compared to 
100-95 = 5 percent. 

If the comparison of average scale scores 
across two years were made for only one of 
the five groups, there would be a significant 
difference between the average scale scores 
for the two years if the significance level 
were less than 5 percent. However, because 



23 Miller, R. G. (1966). Simultaneous statistical inference. New York, NY; Wiley. 

24 Benjaniini,Y, & Hochberg,Y (1995). Controlling the false discovery rate: A practical and powerful approach to 
multiple testing. Jowma/ of the Royal Statistical Society, Series B, No. 1., pp 289—300. 

25 Williams, V. S. L., Jones, L. V, &Tukey,J.W, (1994, December). Controlling error in multiple comparisons with special 
attention to the National Assessment of Educational Progress. Research Triangle Park, NC: National Institute of 
Statistical Sciences. 
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we are interested in the difference in 
average scale scores across the two years for 
all five of the groups, comparing each of 
the significance levels to 5 percent is not 
adequate. Groups of students defined by 
shared characteristics, such as race/ethnicity 
groups, are treated as sets or families when 
making comparisons. However, compari- 
sons of average scale scores for each pair of 
years were treated separately. So the steps 
described in this example would be repli- 
cated for the comparison of other current 
and previous year average scale scores. 

To use the FDR procedure to take into 
account that all comparisons are of interest 
to us, the percents of confidence in the 
example are ordered fi:om largest to smallest: 
62, 35, 20, 4, and 1 . In the FDR procedure, 
62 percent confidence for the Group 4 
comparison would be compared to 5 
percent, 35 percent for the Group 5 
comparison would be compared to 
0.05^(5-1)75 = 0.04 = 4 percent,^" 

20 percent for the Group 1 comparison 
would be compared to 0.05'*'(5-2)/5 = 



0.03 — 3 percent, 4 percent for the Group 
3 comparison would be compared to 
0.05^(5-3)75 = 0.02 = 2 percent, 
and 1 percent for the Group 2 comparison 
(actually slightly smaller than 1 prior to 
rounding) would be compared to 
0.05^(5-4)75 = 0.01 = 1 percent. 

The last of these comparisons is the only 
one for which the percent confidence is 
smaller than the FDR procedure value. The 
difference in the current year and previous 
years’ average scale scores for the Group 2 
students is significant; for all of the other 
groups, average scale scores for current and 
previous year are not significantly different 
from one another. In practice, a very small 
number of counterintuitive results occur 
when using the FDR procedures to exam- 
ine between-year differences in subgroup 
results by jurisdiction. In those cases, results 
were not included in this report. NCES is 
continuing to evaluate the use of FDR and 
multiple-comparison procedures for future 
reporting. 



TableA.12 



FDR comparisons of average scale scores for different groups of students 





Previous year 


Current year 


Previous year and current year 




Average 
scale score 


Standard 

error 


Average 
scale score 


Standard 

error 


Difference 
in averages 


Standard 
error of 
difference 


Test 

statistic 


Percent 

confidence* 


Group 1 


224 


1.3 


226 


1.0 


2.08 


1.62 


1.29 


20 


Group 2 


187 


1.7 


193 


1.7 


6.31 


2.36 


2.68 


1 


Group 3 


191 


2.6 


197 


1.7 


6.63 


3.08 


2.15 


4 


Group 4 


229 


4.4 


232 


4.6 


3.24 


6.35 


.51 


62 


Group 5 


201 


3.4 


196 


4.7 


-5.51 


5.81 


-.95 


35 



* The percent confidence is 2(1 -F(x)) where F(x) is the cumulative distribution of the t-distribution with the degrees of freedom adjusted to reflect the 
complexities of the sample design. 
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The level of confidence times the number of comparisons minus one divided by the number of comparisons is 
0.05*(5-l)/5 = 0.04 = 4 percent. 
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NAEP Reporting Groups 

Results are provided for groups of students 
defined by shared characteristics — region 
of the country, gender, race or ethnicity, 
school’s type of location, eligibility for the 
free/ reduced-price school lunch program, 
and type of school. Based on participation 
rate criteria, results are reported for sub- 
populations only when sufficient numbers 
of students and adequate school representa- 
tion are present. The minimum require- 
ment is at least 62 students in a particular 
subgroup from at least five primary sam- 
pling units (PSUs).^^ However, the data for 
all students, regardless of whether their 



subgroup was reported separately, were 
included in computing overall results. 
Definitions of the subpopulations are 
presented below. 

Region 

Results in NAEP are reported for four 
regions of the nation: Northeast, Southeast, 
Central, and West. Figure A. 2 shows how 
states are subdivided into these NAEP 
regions. All 50 states and the District of 
Columbia are fisted. Other jurisdictions, 
including territories and the two Depart- 
ment of Defense Educational Activities 
jurisdictions are not assigned to any region. 



Figure A.2 


States included in the four NAEP regions 


Northeast 


Southeast 


Central 


West 


Connecticut 


Alabama 


Illinois 


Alaska 


Delaware 


Arkansas 


Indiana 


Arizona 


District of Columbia 


Florida 


Iowa 


California 


Maine 


Georgia 


Kansas 


Colorado 


Maryland 


Kentucky 


Michigan 


Hawaii 


Massachusetts 


Louisiana 


Minnesota 


Idaho 


New Hampshire 


Mississippi 


Missouri 


Montana 


New Jersey 


North Carolina 


Nebraska 


Nevada 


New York 


South Carolina 


North Dakota 


New Mexico 


Pennsylvania 


Tennessee 


Ohio 


Oklahoma 


Rhode Island 


*Virginia 


South Dakota 


Oregon 


Vermont 


West Virginia 


Wisconsin 


Texas 


*Virginia 






Utah 








Washington 








Wyoming 



* The part of Virginia that is included in the Northeast region is the Washington, DC metropolitan area; the remainder of the state is included in the Southeast 
region. 



For the national assessment, a PSU is a selected geographic region (a county, group of counties, or metropolitan 
statistical area). For the state assessment program, a PSU is most often a single school. Further details about the 
procedure for determining minimum sample size appear in the NAEP 1996 Technical Report and the forthcoming 
NAEP 2000 Technical Report, 
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Gender 

Results are reported separately for males 
and females. 

Race/Ethnicity 

The race/ ethnicity variable is derived from 
two questions asked of students and from 
school records, and it is used for race/ 
ethnicity subgroup comparisons. Two 
questions from the set of general student 
background questions were used to deter- 
mine race/ethnicity: 

If you are Hispanic, what is your Hispanic 
background? 

□ l am not Hispanic 

□ Mexican, Mexican American, or Chicano 
Q Puerto Rican 

□ Cuban 

Q Other Spanish or Hispanic background 

Students who responded to this question 
by fiUing in the second, third, fourth, or 
fifth oval were considered Hispanic. For 
students who filled in the first oval, did not 
respond to the question, or provided 
information that was illegible or could not 
be classified, responses to the following 
question were examined to determine their 
race /ethnicity. 



Which best describes you? 

□ White (not Hispanic) 

□ Black (not Hispanic) 

□ Hispanic (“Hispanic” means someone 
who is Mexican, Mexican American, 
Chicano, Puerto Rican, Cuban, or other 
Spanish or Hispanic background) 

□ Asian or Pacific Islander (“Asian or 
Pacific Islander” means someone who is 
from a Chinese, Japanese, Korean, 
Filipino, Vietnamese, Asian American or 
some other Asian or Pacific Islander 
background.) 

□ American Indian or Alaskan Native 
(“American Indian or Alaskan Native” 
means someone who is from one of the 
American Indian tribes or one of the 
original people of Alaska.) 

□ Other (specify) 

Students’ race /ethnicity was then assigned 
on the basis of their responses. For students 
who filled in the sixth oval (“Other”), 
provided illegible information or informa- 
tion that could not be classified, or did not 
respond at all, race/ethnicity was assigned as 
determined by school records. 

Race/ethnicity could not be determined 
for students who did not respond to either 
of the demographic questions and whose 
schools did not provide information about 
race/ethnicity. 
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Also, some students indicated that they 
were from a Hispanic background (e.g., 
Puerto Rican or Cuban) and that a racial/ 
ethnic category other than Hispanic best 
described them. These students were 
classified as Hispanic based on the rules 
described above. 

Type of Location 
Results from the 2000 assessment are 
reported for students attending schools in 
three mutually exclusive location types: 
central city, urban fringe/large town, and 
rural/small town: 

Central City: This category includes central 
cities of all Standard Metropolitan Statisti- 
cal Areas (SMSA) as defined by the Office 
of Management and Budget. Central City 
is a geographical term and is not synony- 
mous with “inner city.” 

Urban Fringe /Large Town: The urban fringe 
category includes all densely settled places 
and areas within SMSA s that are classified 
as urban by the Bureau of the Census, but 
which do not qualify as Central City. A 
Large Town is defined as a place outside a 
SMSA with a population greater than or 
equal to 25,000. 

Rural/Small Town: Rural includes all places 
and areas with populations of less than 
2,500 that are classified as rural by the 
Bureau of the Census. A Small Town is 
defined as a place outside a SMSA with a 
population of less than 25,000, but greater 
than or equal to 2,500. 







Results for each type of location are not 
compared across years. This was due to new 
methods used by NCES to identify the 
type of location assigned to each school in 
the Common Core of Data (CCD). The 
new methods were put into place by 
NCES in order to improve the quality of 
the assignments and they take into account 
more information about the exact physical 
location of the school. 

Eligibility for the Free/Reduced-Price 
School Lunch Program 
Based on available school records, students 
were classified as either currently eligible 
for the free/reduced-price school lunch 
component of the Department of 
Agriculture’s National School Lunch 
Program or not eligible. Eligibility for the 
program is determined by students’ family 
income in relation to the federally estab- 
lished poverty level. Free lunch qualifica- 
tion is set at 130 percent of the poverty 
level, and reduced-price lunch qualification 
is set at 170 percent of the poverty level. 
The classification applies only to the school 
year when the assessment was administered 
(i.e.,the 1999—2000 school year) and is 
not based on eligibility in previous years. 

If school records were not available, the 
student was classified as “Information not 
available.” If the school did not participate 
in the program, all students in that school 
were classified as “Information not available.” 
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Type of School 

Results are reported by the type of school 
that the student attends — pubUc or non- 
public. Nonpublic schools include Catholic 
and other private schools.^* Because they 
are funded by federal authorities, not state/ 
local governments, Bureau of Indian Affairs 
(BIA) schools and Department of Defense 
Domestic Dependent Elementary and 
Secondary Schools (DDESS) are not 
included in either the pubhc or nonpubhc 
categories; they are included in the overaU 
national results. 

Grade 12 Participation Rates and 
Motivation 

NAEP has been described as a “low-stakes” 
assessment. That is, students receive no 
individual scores, and their NAEP perfor- 
mance has no effect on their grades, pro- 
motions, or graduation. There has been 
continued concern that this lack of conse- 
quences affects participation rates of stu- 
dents and schools, as weU as the motivation 
of students to perform weU on NAEP. Of 
particular concern has been the perfor- 
mance of twelfth-graders, who typicaUy 
have lower student participation rates than 
fourth- and eighth-graders, and who are 
more likely to omit responses compared to 
the younger cohorts. 



Participation Rates 

In NAEP, there has been a consistent 
pattern of lower participation rates for 
older students. In the 2000 NAEP assess- 
ments, for example, the student participa- 
tion rates were 96 percent and 92 percent 
at grades 4 and 8, respectively. At grade 12, 
however, the participation rate was 76 
percent. School participation rates (the 
percentage of sampled schools that partici- 
pated in the assessment) have also typicahy 
decreased with grade level. Again citing the 
2000 assessments, the school participation 
rate was 88 percent for the fourth grade, 85 
percent for the eighth grade, and 82 per- 
cent for the twelfth grade. 

The effect of participation rates on 
student performance, however, is unclear. 
Students may choose not to participate in 
NAEP for many reasons, such as desire to 
attend regular classes so as not to miss 
important instruction or conflict with 
other school-based activities. Similarly, 
there are a variety of reasons for which 
various schools do not participate. The 
sampling weights and nonresponse adjust- 
ments, described earher in this document, 
provide an approximate statistical adjust- 
ment for nonparticipation. However, the 
effect of some school and student 
nonparticipation may have some undeter- 
mined effect on results. 



28 Through a pilot study, more detailed breakdowns of nonpublic school results ate available on the NAEP web site 

(http://nces.ed.gov/nationsreportcard/science/results/index.asp). 
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Motivation 

To the extent that students in the NAEP 
sample are not trying their hardest, NAEP 
results may underestimate student perfor- 
mance. The concern increases as students 
get older, and may be particularly pro- 
nounced for twelfth-graders. The students 
themselves furnish some evidence about 
their motivation. As part of the background 
questions, students were asked how impor- 
tant it was to do well on the NAEP science 
assessment. They were asked to indicate 
whether it was very important, important, 
somewhat important, or not very impor- 
tant to them. The percentage of students 
indicating they thought it was either 
important or very important to do well was 
89 percent for fourth-graders, 58 percent 
for eighth-graders, and 31 percent for 
twelfth-graders. 

It is also interesting to note that students 
who indicated it was very important for 
them to do well on NAEP did not have 
the highest average scores. In fact, at grades 
8 and 12, students who reported it was not 
very important to do well also had higher 
average scores than those who reported it 
was very important to do well. These data 
further cloud the relationship between 
motivation and performance on NAEP. 



Need for Future Research 

More research is needed to delineate the 
factors that contribute to nonparticipation 
and lack of motivation. To that end, NCES 
is currently investigating how various types 
of incentives can be eflfectively used to 
increase participation in NAEP. One report 
that examines the impact of monetary 
incentives on student eflFort and perfor- 
mance is available on the NCES web site at 
http://nces.ed.gov/pubsearch/. Enter 
NCES#: 2001024. 

Cautions in Interpretations 

As described earher, the NAEP science 
scale makes it possible to examine relation- 
ships between students’ performance and 
various background factors measured by 
NAEP. However, a relationship that exists 
between achievement and another variable 
does not reveal its underlying cause, which 
may be influenced by a number of other 
variables. Similarly, the assessments do not 
capture the influence of unmeasured 
variables. The results are most useful when 
they are considered in combination with 
other knowledge about the student popu- 
lation and the educational system, such as 
trends in instruction, changes in the 
school-age population, and societal de- 
mands and expectations. 
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Data Appendix 



Appendix 

Focus 



Complete data 
for all tables 
and figures. 



This appendix contains complete data for all the tables and 
figures presented in this report, including average scores, 
achievement-level results, and percentages of students. In 
addition, standard errors appear in parentheses next to each 
scale score and percentage. The comparisons presented in 
this report are based on statistical tests that consider the 

magnitude of the difference between group averages 
or percentages and the standard errors of those 
statistics. Because NAEP scores and percentages are 
based on samples rather than the entire population(s), 
the results are subject to a measure of uncertainty 
reflected in the standard errors of the estimates. It can 
be said with 95 percent certainty that for each 
population of interest, the value for the whole 
population is approximately within plus or minus two 
standard errors of the estimate for the sample. As with 
the figures and tables in the chapters, significant 
differences between results of previous assessments 
and the 2000 assessment are highhghted. 
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Overall percentage correct and percentages correct Avithin each achievement-level range: 2000 





a@0oS@?iaii^ 


Overall percentage 
correct^ 

55(1,5) 


■ — ■■ i 

Basic ' Proficient Advanced 

' 138-1B9* 170-204* 205 and above* 

55(3.1) ' 75 (2.7) 90 (4.7) 



Mncludes fourth-grade students who were below the Basic level. 

*NAEP Science composite scale range, 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 




Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 

2000 



®ja(ite4) 




Overall percentage 


; Basic 


! 

Proficient 


Advanced 


*Xomplete” ^ 


i 138-1B9* 

i 


170-204* 

i 


205 and above* 


28(1,5) 


26 (2.3) 


1 45 (3.8) 


65(12.2) 



includes fourth-grade students who were below the Basic level, 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 




Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 

2000 



®ja(ite4) 




Overall percentage 


Basic 


Proficient 


Advanced 


“Complete” ^ 


j 138-169* 

1 


170-204* 


205 and above* 


5 (0,5) 


1 4 (0.9) 


10(2.1) 

1 


23 (7,6) 



^Includes fourth-grade students who were below the Basic level, 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment, 
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Overall percentage correct and percentages correct within each achievement-level range: 2000 



®ca(!l@© 


a(§Oo0(3Mi](5^ 


Overall percentage 


Basic 1 


Proficient 


Advanced 


correct * 


14J-169* 1 


1 17(^-207* 


208 and above* 


59(1.5) 


’ 59(3.1) ! 

! J 


71 (2.7) 


81(6.1) 



’Includes eighth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 




Overall percentage ‘"Complete” and percentages “Complete” within each achievement-level range: 

2000 



(Mi(!l@S 


S(i)0(Mgii)@(i]B4^ 


Overall percentage 
“Complete” ^ 


Basic j 

148-169* j 


Proficient 

170-207* 


Advanced 
208 and above* 


28(1.3) 


: 28(2.8) 1 


47 (2.8) 


71 (8.7) 



’Includes eighth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 




Overall percentage “Essential” or better and percentages “Essential” or better within each achievement- 
level range: 2000 



(MriteS 




Overall percentage 


Basic 


Proficient 


Advanced 


'‘Essential” or better^ 


\ 148-169* 1 

\ 1 


170-207* 


208 and above* 


24 (1.0) 


23(2.4) 


40 (2.9) 


67 (7.9) 



’Includes eighth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Overall percentage correct and percentages correct within each achievement-level range: 2000 







Overall percentage 


Basic 


Proficient 


Advanced 


correct ^ 


146-177* 


178-209* 


210 and above* 


41(1.3) 


43 (2.6) 


60(3.3) 


75(10.6) 



includes twelfth>grade students who were below the Basic level. 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 




Overall percentage “Essential” or better and percentages “Essential” or better within each achievement- 
level range: 2000 







Overall percentage 
“Essentiar' or better 

19(1.5) 


Basic Proficient Advanced 

146-177* 178-209* 210 and above* 

j ! 

1 18(1.9) 58(4.3) 89(7.7) 



includes twelfth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 




Overall percentage “Essential” or better and percentages “Essential” or better within each achievement 
level range: 2000 





a@OoO(?waii)@(ffi 


Overall percentage 
“Essential" or better 

22(1.5) 


Basic Proficient Advanced 

146-177* 178-209* 210andabove* 

24(2.7) 44(3.7) 56(13.7) 



includes twelfth-grade students who were below the Basic level. 

*NAEP Science composite scale range. 

Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2000 Science Assessment. 
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Table B.10; Data for Figure 2.1 National Scale Score Results 



National average science scale scores, grades 4, 8, and 12: 1996 and 2000 





Grade 4 


Grade 8 


Grade 12 


1996 


150 (0.8) 


150 (0.9) 


150 (0.9) 


2000 


150 (0.7) 


151 (0.6) 


147 (1,0) 



Standard errors of the estimated scale scores appear in parentheses. 

* Significantly different from 2000. 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Table 6.11: Data for Figure 2.2: National Performance Distribution 



National science scale score percentiles, grades 4, 8, and 12: 1996 and 2000 







Mean 


10th 


25th 


50th 


75th 


90th 


Grade 4 


1996 


150 (0.8) 


105 (2.2) 


130(1.2) 


153 (0.9) 


173 (0.7) 


190(1.7) 




2000 


150 (0.7) 


105(1.2) 


129 (0.7) 


153 (0.8) 


174 (0.8) 


191 (0.8) 


Grade 8 


1996 


150 (0.9) 


104(1.0) 


128(1.0) 


153(1.2) 


174(1.5) 


192(1.5) 




2000 


151 (0.6) 


103(1.2) 


128(0.8) 


154(1.0) 


177 (0.8) 


195 (0.6) 


Grade 12 


1996 


150 (0.9)* 


104(1.1) 


128(1.2) 


152(1.1)* 


174(1.2) 


191(1.2) 




2000 


147(1.0) 


102(1.2) 


125(1.1) 


148(1.0) 


171(1.3) 


190(1.1) 



Standard errors of the estimated scale scores appear in parentheses. 

* Significantly different from 2000. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table B.12: Data for Figure 2.3: National Achievement-Level Results 



Percentage of students within each science achievement-level range and at or above 
achievement levels, grades 4, 8, and 12: 1996 and 2000 



r— 

' Below Basic At Basic 



Grade 4 


1996 


33 (1.2) 


38 (0.8) 




2000 , 


34 (0.8) 


37 (0.7) 


Grade 8 


1996 


39(1.1) 


32(0.7)* * 




2000 


39 (0.8) 


29 (0.5) 


Grade 12 


1996 : 


43 (1.1) * 


36(1.0) 




2000 ; 


47 (1.1) 


34 (0.7) 



At Proficient 


Advanced 


M or stove ' 
taste 


At or above 
Proficient 


26 (0.9) 
26 (0.7) 


3 (0.4) 

4 (0.3) 


67 (1.2) 
66 (0.8) 


29 (0.9) 
29 (0.8) 


26(1.1) 
28 (0.7) 


3 (0.5) 

4 (0.4) 


61(1.1) 
61 (0.8) 


29(1.2)* 
32 (0.8) 


19(1.0) 

16(0.9) 


3 (0.3) 
2 (0.3) 


57(1.1)* 

53(1.1) 


21(1.1) 

18(1.0) 



standard errors of the estimated percentages appear in parentheses. 

* Significantly different from 2000. 

NOTE: Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 



Table B.13: Data for Figure 2.4 National Scale Score Results by Region 



Percentage of students and average science scale score results by region of the country, 
grades 4, 8, and 12: 1996 and 2000 







Northeast 


Southeast 


Central 


West 


Grade 4 


1996 


22(1.5) 

156(1.8) 


22(1.7) 
143 (2.0) 


26(1.3) 
156 (2.1) 


30(1.9) 
146 (2.0) 




2000 


23 (0.8) 
153(1.1) 


23(1.2) 
143 (1.7) 


24 (0.5) 
155(1.8) 


30(1.2) 

148(1.5) 


Grade 8 


1996 


22(1.6) 
151 (2.6) 


22 (2.2) 
143(1.9) 


24 (0.5) 
156 (2.5) 


32 (2.4) 
149 (2.2) 




2000 


22 (0.5) 
153(1.5) 


21 (0.5) 
145 (1.4) 


25 (0.4) 
158(1.6) 


32 (0.7) 
148(1.3) 


Grade 12 


1996 


22(1.3) 
154 (2.8) 


21(1.8) 

142(1.4) 


24 (0.9) 
158 (2.0) * 


33(1.8) 
147 (2.3) 




2000 


21(1.2) 
151 (2.9) 


22(1.5) 
141 (1.6) 


26 (0.5) 
150(1.7) 


32(1.4) 

145(1.9) 



The percentage of students is listed first with the corresponding average scale score presented below. 

Standard errors of the estimated percentages and scale scores appear in parentheses. 

* Significantly different from 2000. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics. National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table B.14: Data for Figure 2.5: National Achievement-Level Results by Region 



Percentage of students within each science achievement-level range and at or above 
achievement levels, by region of the country, grades 4, 8, and 12: 1996 and 2000 



M or above ' At or above 









Below B$sic 


At Basic 


At Proficient 


l\X Advanced 




Proficient 


Grade 4 


Northeast 


1996 


26 (2.4) 


39(1.6) 


31(2.3) 


4 (0.9) 


74 (2.4) 


36(2.3) 






2000 


30 (1.4) 


38(1.3) 


28(1.9) 


4 (0.8) 


70(1.4) 


32(1.7) 




Southeast 


1996 


43 (2.5) 


34(1.6) 


21 (1.5) 


2 (0.5) 


57 (2.5) 


23(1.8) 






2000 


41 (1.9) 


35(1.6) 


21 (1.3) 


2 (0.5) 


59(1.9) 


23(1.2) 




Central 


1996 


26 (3.2) 


41(2.5) 


30 (2.0) 


4 (0.6) 


74 (3.2) 


34 (2.0) 






2000 


27 (1.9) 


39 (2.0) 


30 (2.1) 


5 (0.9) 


73(1.9) 


35 (2.4) 




West 


1996 


37 (3.0) 


39(1.7) 


21 (2.0) 


2 (0.6) 


63 (3.0) 


24 (2.2) 






2000 


36 (1.8) 


36(1.5) 

j 


25(1.7) 


3 (0.5) 


64(1.8) 


28(1.8) 


Grade 8 


Northeast 


1996 1 


38 (3.2) 


32(1.9) 


27 (2.7) 


3(1.1) 


62 (3.2) 


30 (3.5) 






2000 ! 


37 (1.8) 


28(1.6) 


30(1.7) 


5 (0.9) 


63(1.8) 


35(1.9) 




Southeast 


1996 


47 (2.6) 


I 30(1.4) 


21(1.7) 


2 (0.3) * 


53 (2.6) 


22(1.8) 






2000 


45(1.8) 


i 29(1.1) 

1 


23(1.3) 


3 (0.5) 


55(1.8) 


26(1.4) 




Central 


1996 


1 32 (2.8) 


< 33(1.7) 


31 (2.7) 


5(1.1) 


68 (2.8) 


35 (3.1) 






2000 


31 (2.2) 


i 31 (1.8) 


34(1.5) 


5 (0.8) 


69 (2.2) 


38(1.9) 




West 


1996 


39 (2.5) 


' 34(1.3)* 


25 (2.0) 


3(1.0) 


61 (2.5) 


28 (2.3) 






2000 


43 (1.6) 


' 28(1.1) 

1 


25(1.3) 


3 (0.5) 


57(1.6) 


29(1.3) 


Grade 12 


Northeast 


1996 


1 40(3.5) 


j 34 (1.6) 


22 (2.4) 


4(0.8) 


60 (3.5) 


26 (2.8) 






2000 


1 43 (3.6) 


i 34 (2.4) 


19(2.2) 


4 (0.9) 


57 (3.6) 


23 (3.0) 




Southeast 


1996 


53(1.8) 


j 33(1.5) 


13(1.2) 


1 (0.4) 


47(1.8) 


14(1.3) 






2000 


1 54(1.9) 


! 32(1.2) 


13(1.2) 


1 (0.3) 


46(1.9) 


14(1.2) 




Central 


1996 


33 (2.4) * 


1 39 (2.1) 


24(2.1) 


4(0.6) 


67 (2.4) * 


28 (2.2)* 






2000 


44 (2.2) 


1 37(1.5) 


17(1.9) 


2(0.7) 


56 (2.2) 


19(1.9) 




West 


1996 


46 (2.8) 


1 37 (2.3) 


15(1.7) 


2 (0.7) 


54 (2.8) 


17(2.3) 






2000 


49 (2.1) 


1 34(1.3) 


16(1.3) 


2 (0.4) 


51 (2.1) 


18(1.5) 



standard errors of the estimated percentages appear in parentheses. 

* Significantly different from 2000. 

NOTE; Percentages within each science achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table B.1 5: Data for Table 2.1 State Average Score Results, Grade 4 



Average science scale score results by state for grade 4 public schools: 2000 



Nation 


148 (0.8) 


Alabama 


143(1.7) 


Arizona 


141(1.4) 


Arkansas 


144(1.7) 


California ^ 


131 (2.0) 


Connecticut 


156(1.3) 


Georgia 


143(1.4) 


Hawaii 


136(1.4) 


Idaho ^ 


153(1.5) 


Illinois ^ 


151(1.6) 


Indiana ^ 


155(1.6) 


Iowa ^ 


160(1.4) 


Kentucky 


152(1.1) 


Louisiana 


139(1.9) 


Maine ^ 


161(1.0) 


Maryland 


146(1.3) 


Massachusetts 


162(1.2) 


Michigan ^ 


154(1.8) 


Minnesota ^ 


157(1.5) 


Mississippi 


133(1.4) 


Missouri 


156(1.6) 


Montana ^ 


160 (2.1) 


Nebraska 


150(1.8) 


Nevada 


142(1.3) 


New Mexico 


138 (2.0) 


New York ^ 


149(1.4) 


North Carolina 


148(1.4) 


North Dakota 


160 (0.8) 


Ohio ^ 


154(1.6) 


Oklahoma 


152(1.4) 


Oregon ^ 


150(1.9) 


Rhode Island 


148(1.5) 


South Carolina 


141(1.2) 


Tennessee 


147(1.5) 


Texas 


147(1.6) 


Utah 


155(1.1) 


Vermont ^ 


159(1.7) 


Virginia 


156(1.6) 


West Virginia 


150(1.1) 


Wyoming 


158(1.1) 


Other jurisdictions 




American Samoa 


51(1.7) 


DDESS 


157 (0.7) 


DoDDS 


156(0.5) 


Guam 


110(2.3) 


Virgin Islands 


116(1.1) 



Standard errors of the estimated scale scores appear in parentheses. 

Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: National results are based on the national sample, not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table B.16: Data for Table 2.2 State Average Score Results, Grade 8 



Average science scale score 


results by state 


for grade 8 public schools; 1996 and 2000 




1996 


2000 


Nation 


148 (0.9) 


149 (0.7) 


Alabama 


139(1.6) 


141(1.9) 


Arizona ^ 


145(1.6) 


146(1.6) 


Arkansas 


144(1.3) 


143(1.3) 


California ^ 


138(1.7)* 


132(1.5) 


Connecticut 


155(1.3) 


154(1.4) 


Georgia 


142(1.4) 


144(1.5) 


Hawaii 


135 (0.7) 


132(1.2) 


Idaho 


— 


159(1.1) 


Illinois ^ 


— 


150(1.9) 


Indiana ^ 


153(1.4) 


156(1.7) 


Kentucky 


147(1.2)* 


152(1.3) 


Louisiana 


132(1.6) 


136(1.7) 


Maine ^ 


163(1.0)* 


160(1.0) 


Maryland 


145(1.5) 


149(1.3) 


Massachusetts 


157(1.4) 


161(1.6) 


Michigan ^ 


153(1.4) 


156(1.7) 


Minnesota ^ 


159(1.3) 


160 (2.1) 


Mississippi 


133(1.4) 


134(1.2) 


Missouri 


151(1.2)* 


156(1.1) 


Montana ^ 


162(1.2) 


165(1.2) 


Nebraska 


157(1.0) 


157(1.0) 


Nevada 


— 


143(1.1) 


New Mexico 


141(1.0) 


140(1.6) 


New York ^ 


146(1.6) 


149 (2.4) 


North Carolina 


147(1.2) 


147 (1.5) 


North Dakota 


162 (0.8) 


161 (0.9) 


Ohio 


— 


161 (1.5) 


Oklahoma 


— 


149(1.2) 


Oregon ^ 


155(1.6) 


154(1.6) 


Rhode Island 


149 (0.8) 


150(1.3) 


South Carolina 


139(1.5) 


142 (1.3) 


Tennessee 


143(1.8) 


146 (1.5) 


Texas 


145(1.8) 


144(1.5) 


Utah 


156(0.8) 


155 (0.9) 


Vermont ^ 


157(1.0)* 


161 (0.9) 


Virginia 


149(1.6) 


152(1.2) 


West Virginia 


147 (0.9) 


150(1.1) 


Wyoming 


158 (0.6) 


158(1.0) 


Other jurisdictions 


American Samoa 


— 


72 (2.3) 


DDESS 


153(1.1)* 


159(1.2) 


DoDDS 


155 (0.7)* 


159 (0.8) 


Guam 


120(1.1) 


114(4.5) 



Standard errors of the estimated scale scores appear in parentheses. 

* Significantly different from 2000 if only one jurisdiction or the nation is being examined. 

^ Significantly different from 2000 when examining only one jurisdiction and when using a multiple comparison procedure based on all jurisdictions that 
participated both years. 

^ Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2000. 

— Indicates that the jurisdiction did not participate. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. DoDDS: Department of Defense Dependents Schools (Overseas). 
NOTE: National results are based on the national sample, not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 and 2000 Science Assessments. 
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Table B.17: Data for Figure 2.10 State Achievement-Level Results, Grade 4 



Percentage of students within each science achievement-level range by state for grade 4 public 
schools: 2000 





Below Basic 


At Basic 


At Proficient 


At Advanced 


Nation 


36 (0.9) 


37 (0.7) 


24 (0.8) 


3 (0.3) 


Alabama 


41(2.1) 


37(1.5) 


20(1.2) 


2 (0.4) 


Arizona 


43(1.7) 


35(1.6) 


20(1.4) 


2 (0.4) 


Arkansas 


38 (2.1) 


38 (1.4) 


22(1.5) 


2 (0.6) 


California ^ 


53 (2.4) 


33(1.8) 


13(1.5) 


1 (0.2) 


Connecticut 


25(1.7) 


40(1.2) 


32(1.6) 


3 (0.6) 


Georgia 


42(1.8) 


34(1.4) 


21(1.2) 


3 (0.4) 


Hawaii 


49(1.5) 


35(1.1) 


15(1.0) 


1 (0.3) 


Idaho ^ 


28 (2.1) 


42(1.7) 


28(1.9) 


3 (0.6) 


Illinois ^ 


32 (2.1) 


37(1.8) 


27(2.1) 


4 (0.9) 


Indiana ^ 


25(1.9) 


42 (1.4) 


29(1.8) 


3 (0.5) 


Iowa ^ 


19(2.2) 


44(1.6) 


34(1.8) 


4 (0.6) 


Kentucky 


30 (1.5) 


42(1.5) 


26(1.4) 


3 (0.4) 


Louisiana 


47 (2.3) 


35(1.7) 


17(1.7) 


2 (0.4) 


Maine ^ 


18(1.4) 


43(1.6) 


34(1.6) 


4 (0.7) 


Maryland 


39(1.5) 


36(1.2) 


23(1.4) 


3 (0.5) 


Massachusetts 


19(1.4) 


38(1.6) 


37(1.7) 


6 (0.7) 


Michigan ^ 


29 (2.1) 


38(1.7) 


30 (2.1) 


3 (0.7) 


Minnesota ^ 


22(1.8) 


42(1.5) 


32 (2.0) 


3 (0.5) 


Mississippi 


53(1.9) 


33(1.3) 


13(1.1) 


1 (0.3) 


Missouri 


25(1.8) 


40(1.5) 


31(1.6) 


4 (0.5) 


Montana ^ 


19(2.7) 


44 (2.6) 


34 (2.8) 


4 (0.8) 


Nebraska 


32 (2.1) 


41 (2.0) 


24 (2.0) 


2 (0.8) 


Nevada 


42(1.9) 


39(1.6) 


18(0.9) 


2 (0.4) 


New Mexico 


46 (2.3) 


36(1.9) 


16(1.1) 


2 (0.6) 


New York ^ 


33 (2.0) 


41(1.5) 


24(1.2) 


2 (0.3) 


North Carolina 


36(1,9) 


40(1.7) 


22(1.3) 


2 (0.5) 


North Dakota 


20(1,2) 


43(1.3) 


34(1.2) 


3 (0.5) 


Ohio^ 


28(1,9) 


40(1.7) 


28(1.5) 


4 (0.7) 


Oklahoma 


29 (2,1) 


45 (2.0) 


24(1.8) 


2 (0.4) 


Oregon ^ 


33 (2,4) 


40(1.7) 


25(1.6) 


3 (0.7) 


Rhode Island 


34 (2,0) 


40 (1.4) 


24(1.3) 


2 (0.4) 


South Carolina 


44(1.5) 


35(1.7) 


18(1.1) 


2 (0.3) 


Tennessee 


37(1.9) 


38(1.7) 


23(1.5) 


3 (0.5) 


Texas 


35 (2.1) 


40(1.4) 


22(1.5) 


2 (0.4) 


Utah 


25(1.5) 


43(1,1) 


29(1.1) 


3 (0.5) 


Vermont ^ 


22(1,9) 


40 (2.4) 


34 (2.6) 


4(1.1) 


Virginia 


26(1,9) 


41(1.6) 


29(1.8) 


4 (0.6) 


West Virginia 


31(1,7) 


45(1.6) 


23(1.3) 


2 (0.3) 


Wyoming 


20(1,9) 


47(1.7) 


30(1.5) 


3 (0.5) 


Other Jurisdictions 


American Samoa 


98 (0,9) 


2 (0.9) 




0 (****) 


DDESS 


22(1.3) 


48 (2.1) 


27(1.8) 


2 (0.5) 


DoDDS 


25 (0,8) 


45 (1.0) 


27(1.0) 


3 (0.4) 


Guam 


76 (2.0) 


20 (1.6) 


4 (0.9) 




Virgin islands 


74(1.9) 


22 (2.0) 


4 (0.8) 





ERIC 



Standard errors of the estimated percentages appear in parentheses. 

(****) Standard error estimates cannot be accurately determined. 

Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

A Percentage is between 0.0 and 0.5. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Percentages within each science achievement-level range may not add to 100 due to rounding. 

National results are based on the national sample and not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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Table B.18: Data for Figure 2.11 State Achievement-Level Results, Grade 8 



Percentage of students within each science achievement-level range by state for grade 8 public 
schools: 2000 





Below Basic 


At Basic 


At Proficient 


At Advanced 


Nation 


41(0.9) 


29 (0.6) 


26 (0.8) 


4 (0.4) 


Alabama 


49 (2.2) 


29(1.4) 


20(1.4) 


2 (0.5) 


Arizona ^ 


43 (2.2) 


33 (1.6) 


22(1.5) 


2 (0.4) 


Arkansas 


46(1.6) 


31 (1.4) 


21(1.3) 


2 (0.4) 


California ^ 


60 (2.2) 


25(1.9) 


14(1.2) 


1 (0.4) 


Connecticut 


35(1.8) 


30(1.4) 


31(1.4) 


4 (0.6) 


Georgia 


48 (2.0) 


29(1.3) 


21(1.4) 


2 (0.6) 


Hawaii 


60(1.3) 


25(1.5) 


14(1.0) 


1 (0.3) 


Idaho ^ 


27(1.2) 


35(1.1) 


35(1.6) 


4 (0.5) 


Illinois ^ 


38 (2.5) 


31(1.8) 


27(1.8) 


3 (0.8) 


Indiana ^ 


32 (2.3) 


34(1.6) 


31(1.7) 


3 (0.6) 


Kentucky 


38(1.8) 


33(1.5) 


26(1.4) 


3 (0.4) 


Louisiana 


55(2.1) 


m\.i) 


16(1.2) 


2 (0.4) 


Maine ^ 


25(1.3) 


38(1.5) 


33(1.7) 


3 (0.4) 


Maryland 


41(1.7) 


31(1.3) 


26(1.4) 


3 (0.4) 


Massachusetts 


26 (2.0) 


32(1.6) 


37(1.8) 


5 (0.6) 


Michigan ^ 


31 (2.0) 


32(1.3) 


33 (2.0) 


4 (0.8) 


Minnesota ^ 


27 (2.5) 


32 (2.2) 


37 (2.0) 


5 (0.8) 


Mississippi 


58(1.5) 


27(1.3) 


14(1.1) 


1 (0.3) 


Missouri 


32(1.5) 


32(1.5) 


32(1.5) 


4 (0.5) 


Montana ^ 


20(1.7) 


34(1.7) 


41(1.7) 


5 (0.8) 


Nebraska 


30(1.6) 


34(1.7) 


33(1.5) 


4 (0.5) 


Nevada 


46(1.4) 


31(1.4) 


21(1.2) 


2 (0.3) 


New Mexico 


52(1.9) 


28(1.9) 


19(1.4) 


1 (0.3) 


New York ^ 


39(2.7) 


32(1.6) 


27(2.1) 


2 (0.6) 


North Carolina 


44(1.9) 


30(1.4) 


23(1.3) 


3 (0.6) 


North Dakota 


26(1.2) 


34(1.5) 


36(1.9) 


4 (0.7) 


Ohio 


27(1.8) 


32(1.3) 


35(1.8) 


6 (0.7) 


Oklahoma 


38(1.5) 


35(1.4) 


25(1.3) 


2 (0.4) 


Oregon ^ 


33 (2.1) 


34(1.9) 


30(1.7) 


3 (0.6) 


Rhode Island 


39(1.3) 


32(1.3) 


26(1.1) 


3 (0.4) 


South Carolina 


50(1.8) 


29(1.2) 


18(1.4) 


2 (0.3) 


Tennessee 


43 (2.2) 


32(1.5) 


23(1.3) 


2 (0.4) 


Texas 


47(1.9) 


30(1.3) 


21(1.4) 


2 (0.4) 


Utah 


32(1.2) 


34(1.3) 


31(1.4) 


3 (0.5) 


Vermont ^ 


26(1.5) 


34(1.5) 


35(1.4) 


4 (0.7) 


Virginia 


37(1.6) 


32(1.2) 


28(1.2) 


3 (0.6) 


West Virginia 


39(1.5) 


34(1.4) 


24(1.3) 


2 (0.3) 


Wyoming 


29(1.6) 


35(1.2) 


32(1.1) 


3 (0.5) 


Other Jurisdictions 


American Samoa 


95(1.2) 


3(1.0) 


2 (0.7) 


0 (****) 


DDESS 


30(1.9) 


35 (2.0) 


31(2.1) 


4(1.0) 


DoDDS 


28(1.2) 


34(1.2) 


34(1.3) 


4 (0.8) 


Guam 


78 (2.7) 


16 (2.3) 


6(1.4) 





Standard errors of the estimated percentages appear in parentheses. 

(****) Standard error estimates cannot be accurately determined. 

Indicates that the jurisdiction did not meet one or more of the guidelines for school participation. 

A Percentage is between 0.0 and 0.5. 

DDESS: Department of Defense Domestic Dependent Elementary and Secondary Schools. 

DoDDS: Department of Defense Dependents Schools (Overseas). 

NOTE: Percentages within each science achievement-level range may not add to 100 due to rounding. 

National results are based on the national sample and not on aggregated state assessment samples. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment. 
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